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Preface to the third volume of Model Order Reduction 


The third volume of the Model Order Reduction handbook project offers several re- 
markable instances of applications of model order reduction (MOR) approaches to the 
solution of problems arising from the most diverse areas of application. Through these 
examples, we would like to provide the reader with an overview of the maturity of this 
emerging field and its readiness to address challenging problems of multifaceted com- 
plexity. 

We start with several chapter contributions to classical fields of engineering. 

The first one, by J. Eason and L. Biegler, is on model reduction in the optimization 
of a variety of heterogeneous chemical processes. In particular, two case studies are 
presented on CO, capture using nonlinear programming and NLP filter models. 

The second chapter, by B. Lohmann etal., is on MOR in mechanical engineering. 
Four applications are discussed, concerning the reduction of a thermo-mechanical 
machining tool of a car body and driver’s seat, of an elastic crankshaft, and a leaf 
spring model. 

The third chapter, by E. Deckers etal., presents several case studies of MOR for 
acoustics and vibrations in mechanical applications. Two different viewpoints are de- 
veloped: the application of MOR from a purely mathematical perspective and a con- 
sideration of expected properties of MOR based on physical arguments from the field 
of mechanics. 

Two chapters devoted to microelectronics and electromagnetism, a very classical 
and successful arena for MOR methods, follow. The first of those, by B. Nouri etal., 
pursues a twofold goal: to describe the context in which the need for MOR arose in 
microelectronics, and to present an overview of their applications to address the is- 
sues of high-speed interconnects in microelectronics at various levels of the design 
hierarchy. 

The next chapter, by D. Ioan et al., proposes a computer-aided consistent and ac- 
curate description of the behavior of electromagnetic devices at various speeds or fre- 
quencies, and describes procedures to generate compact electrical circuits featuring 
an approximately equivalent behavior. 

The chapter by M. Yano is on model reduction in computational aerodynamics. 
The focus is on techniques that are designed to address nonlinearity, limited stability, 
limited regularity, and a wide range of scales that have been demonstrated successful 
for multidimensional large-scale aerodynamic flows. 

The next two chapters address a somehow less conventional field of applications, 
that of life sciences. The chapter by B. Karas6zen is on MOR in neurosciences, more 
specifically on the exploitation of models of large-scale neuronal networks to provide 
an accurate and fast prediction of patterns and their propagation in different areas of 
the brain. 

The following chapter, by N. Dal Santo etal., introduces MOR methods to face 
some of the most challenging processes of the cardiovascular system. Two specific 
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applications are targeted: the simulation of blood flow past a carotid bifurcation and 
the computation of activation maps in cardiac electrophysiology. 

The last five chapters address somewhat more methodological issues arising in 
various scientific, engineering, societal, and economics applications. 

The chapter by J.-C. Loiseau aims at bypassing some difficulties of classical proper 
orthogonal decomposition approaches to the solution of fluid dynamics problems by 
using feature-based manifold modeling in which the low-dimensional attractor and 
nonlinear dynamics are characterized from experimental data: time-resolved sensor 
data and optional nontime-resolved particle image velocimetry snapshots. 

In the chapter by R. Pulch, MOR is used in the framework of uncertainty quan- 
tification. Established methods like polynomial chaos, stochastic Galerkin, stochastic 
collocation, and quadrature sampling are reviewed for dynamical systems consisting 
of ordinary differential equations or differential algebraic equations. Demonstration 
of applicability is provided on test examples. 

The chapter by X. Cheng et al. addresses MOR methods for networks that describe 
a wide class of complex systems composed of many interacting subsystems. First, 
clustering-based approaches are reviewed, with the aim of reducing the network scale. 
Then, methods based on generalized balanced truncation that reduce interconnection 
structures of a network and the dynamics of each subsystem are discussed. 

The chapter by D. Hartmann etal. presents use cases where MOR is a key enabler 
for the realization of digital services and the reduction of simulation times and out- 
lines the potential of MOR in the context of realizing the digital twin vision. 

The last chapter, by B. Haasdonk, addresses the issue of software. In the first part, 
as neither full simulation models nor MOR algorithms are to be reprogrammed, but 
ideally are reused from existing implementations, the interplay of such packages is 
discussed. Then an overview of the most popular MOR software libraries is provided. 

We are confident that the vast set of applications discussed here, combined with 
the broad variety of numerical techniques and software libraries available, will moti- 
vate the reader to embrace MOR approaches to successfully address complex applica- 
tions arising in computational science and engineering. 
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John P. Eason and Lorenz T. Biegler 
1 Model reduction in chemical process 
optimization 


Abstract: Chemical processes are often described by heterogeneous models that range 
from algebraic equations for lumped parameter systems to black-box models for PDE 
systems. The integration, solution, and optimization of this ensemble of process mod- 
els is often difficult and computationally expensive. As a result, reduction in the form 
of reduced-order models and data-driven surrogate models is widely applied in chem- 
ical processes. This chapter reviews the development and application of reduced mod- 
els (RMs) in this area, as well as their integration to process optimization. Special at- 
tention is given to the construction of reduced models that provide suitable represen- 
tations of their detailed counterparts, and a novel trust region filter algorithm with 
reduced models is described that ensures convergence to the optimum with truth mod- 
els. Two case studies on CO, capture are described and optimized with this trust re- 
gion filter method. These results demonstrate the effectiveness and wide applicability 
of the trust region approach with reduced models. 


Keywords: Model reduction, trust region methods, POD, equation-oriented modeling, 
glass box, black box, nonlinear programming, NLP filter methods 


MSC 2010: 35B30, 37M99, 41A05, 65K99, 93A15, 93C05 


1.1 Introduction 


Chemical processes incorporate advanced technologies that need to be modeled, inte- 
grated, and optimized. To address these needs, state-of-the-art nonlinear optimization 
algorithms can now solve models with millions of decision variables and constraints. 
Correspondingly, the computational cost of solving discrete optimization problems 
has been reduced by several orders of magnitude [14]. Moreover, these algorithmic ad- 
vances have been realized through software modeling frameworks that link optimiza- 
tion models to efficient nonlinear programming (NLP) and mixed-integer NLP solvers. 
On the other hand, these advances are enabled through modeling frameworks that 
require optimization models to be formulated as well-posed problems with exact first 
and second derivatives. 

Despite these advances, multiscale processes still need effective problem formu- 
lation and modeling environments. At the process optimization level, multiscale in- 
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tegration is required to model complex transport and fluid flow phenomena. For in- 
stance, optimization models for advanced power generation processes, such as in Fig- 
ure 1.4, comprise a heterogeneous modeling environment with algebraic equation (AE) 
models, such as heat exchangers, compressors, and expanders, as well as large, non- 
linear partial differential AE (PDAE) models. These include multiphase reactor models 
such as fluidized beds, combustors, and gasifiers. Because of the substantial complex- 
ity of the associated model solvers, computational costs for multiscale process opti- 
mization are prohibitive. While equation-oriented flowsheet models may take only a 
few CPU seconds to solve, a PDAE combustion or gasification model alone may require 
many CPU hours or even days [48]. We denote these prohibitive models as truth mod- 
els, which require model reduction. These models often follow a “bottom up” model- 
ing approach, where DAEs or PDAEs derive from fundamental physical laws. Models 
at this higher fidelity can include transport behavior and detailed reaction kinetics, 
which require computationally costly simulations. 

The process flowsheet in Figure 1.4 shows a detailed boiler model, pumps, com- 
pressors, turbines, heat exchangers, and mixing and splitting junctions, and the re- 
sulting model comprises equations that connect process units with process streams, 
conservation laws (mass, momentum, energy) within each unit, constitutive equa- 
tions that describe physical phenomena, including transport behavior, equilibrium 
and reaction kinetics, and physical properties for materials and mixtures. This en- 
semble of PDAEs/DAEs/AEs within a chemical process is typically broader than many 
PDAE models in other domains, where model reduction can proceed in a more struc- 
tured manner. 

Model reduction in chemical process simulation and optimization can be effected 
in a number of ways. These include: 

- Simplifying assumptions of physics-based models by removing terms in PDAEs 
that reflect negligible behavior in time and length scales. Often, these “shortcut” 
models can only be defined (and validated) over limited domains of applicability. 

— Time scale reduction, where dynamic behaviors that are either too fast or too slow 
in the range of interest are eliminated [56]. 

— Data-driven input-output models which are generally unstructured, require few 
assumptions, and lead to general-purpose applications [63]. 


Process optimization with reduced models poses a special challenge as most reduced 
models are interpolative, while optimization requires extrapolation. Generally, we ex- 
pect extrapolative capabilities to be captured by physics-based truth models, as they 
are based on fundamental phenomena and comprise constituent models that have 
been validated from many domains. To develop and preserve these capabilities, inter- 
polative reduced models must be reconstructed and recalibrated with truth models, 
in order to remain consistent over the convergence path of the optimization solver. 
To develop an integrated optimization approach we expect that an RM-based 
strategy is allowed to evaluate and compare information from the truth models to cap- 
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ture relevant multiscale phenomena such as complex fluid flow, particle mechanics, 
and dynamic operation within the process optimization. Moreover, general strategies 
can be applied to create RMs through a number of physics-based or data-driven model 
reductions. To develop this strategy, this chapter considers the properties needed for 
the RM-based optimization framework to converge to the optimum of the original 
system models, as well as the construction of RMs that balance model accuracy with 
computational cost during the optimization. 

The next section briefly reviews developments in model reduction that include 
proper orthogonal decomposition (POD) and data-driven models. Section 1.3 then 
presents trust algorithms based on reduced models; these depend on whether gra- 
dients are available from the truth models or not. Section 1.4 then presents two case 
studies that describe the performance of these methods on large-scale process opti- 
mization problems. Most importantly, both methods work directly with the RM, and 
they also guarantee convergence to the optimum of the truth model. Finally, conclu- 
sions and future work are discussed in Section 1.5. 


1.2 Model reduction for simulation 


Model reduction is a broad topic with contributions from many different research com- 
munities. There has always been a balance between model fidelity and computational 
tractability since the earliest use of computers in chemical engineering. For instance, 
for vapor-liquid equilibrium, which is the basic building block of all process models, 
reduced physical property models are often constructed through simplifying thermo- 
dynamic assumptions. In early studies [8, 17, 50] these reduced models proved very 
effective to accelerate calculations without sacrificing much accuracy. While comput- 
ing hardware has improved substantially since that time, these early works show how 
model reduction can be used to solve problems that otherwise may be intractable. 

Beyond the use of simplifying fundamental assumptions with “shortcut” models, 
general model reduction strategies can be partitioned into two categories: model order 
reduction and data-driven model reduction. 

We classify model order reduction methods as projection-based techniques ap- 
plied to an accessible state-space description of the truth model, with an explicit 
projection applied to reduce the state-space dimension [11]. For fully accessible (i. e., 
equation-based) state-space truth models, system-theoretic model order reduction 
exploits the specific dynamic system structure and includes balanced truncation 
and rational interpolation based on Gramians and transfer functions. These usually 
apply to linear systems, although they have been extended to bilinear systems and 
quadratic-in-state systems. Moreover, they are widely applied in circuit theory, sig- 
nal processing, structural mechanics, and linear, optimal control. A comprehensive 
review of system-theoretic methods can be found in [9]. 
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When the truth model is a large-scale system of DAEs, system-theoretic methods 
can take advantage of that structure. In model-based reduction for chemical engineer- 
ing, the truth model is also exploited to guide the projection steps [56], but system- 
theoretic model order reduction is seldom applied to these models. This is mainly 
because of the high model nonlinearity and limited accessibility of the chemical pro- 
cess model equations, often embedded within “gray-box” procedures. To handle these 
truth models, snapshot-based projection methods, such as reduced basis or POD, are 
applied, with sampled snapshot solutions over the parameter domain and space (or 
time) domains. Among these, POD is the most generally applicable as it relies only on 
snapshots of the underlying simulation code. POD has been demonstrated effectively 
in many areas including fluid dynamics, structural dynamics, thermal modeling, and 
atmospheric modeling. As a result, it is frequently applied for model-based reduction 
of large, nonlinear truth models in chemical engineering. 


1.2.1 Proper orthogonal decomposition 


POD, also known as Karhunen—Loéve decomposition, can reduce large spatially dis- 
tributed models to much smaller models. POD models are formulated by projecting 
the PDAE system onto a set of basis functions, which are themselves generated from 
the numerical solution of the original equations. Applications are numerous, with ex- 
amples including [12, 22, 24, 28, 42, 44, 47, 53, 57, 60, 62, 71, 73, 21]. In addition, many 
studies report the use of POD for optimization. However, the basis functions used in 
POD are typically determined from a finite set of simulations of the full-scale PDAE 
system. This greatly reduces the system size, but the accuracy of the POD approxi- 
mation is inherently local in nature. Therefore optimization may have a tendency to 
extrapolate far from data or otherwise exploit approximation errors to find artificially 
improved solutions. 

Nevertheless, several studies report successful use of model order reduction in 
optimization and control. Examples in chemical processes include optimization of 
diffusion-reaction processes [5], transport-reaction processes [10], chemical vapor de- 
position [66], and thermal processing of foods [6]. 

As detailed in [67], POD models are constructed from Galerkin projections of the 
PDAE model onto a set of basis functions. These basis functions are often generated 
empirically from numerical solutions of the truth model, through the method of snap- 
shots. The aim is to find a low-dimensional basis that can capture most information 
of the spatial distribution. To do so, one first gathers snapshot sets which consist of 
spatial solutions of the original PDAE system at several time instants as determined 
by numerical simulation. Let the snapshot matrix be given as 


Z = {z(é,t,),...,2(8, ty, )} (1.1) 
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where each snapshot z(¢, t;) is a column vector representing the (discretized) spatial 
profile at time ¢;. There are N, snapshots and Nọ spatial discretization nodes. 

After gathering a set of snapshots, the singular value decomposition of the snap- 
shot matrix Z is given as 


Z = UDV” = Y ou; . (1.2) 
i 


The first M vectors fu where M < N, E of the orthogonal matrix U represent the 
desired set of POD basis functions (or basis vectors in the case of discretized spatial 
dimensions). From this point we refer to these basis functions as ġ;(¥), since each one 
describes the behavior in the spatial dimensions. 

To determine the number of basis vectors M, the projection error can be approxi- 
mated as 


&(M)= X of. (1.3) 


Then, the interrelation between accuracy and dimension of the POD-based reduced- 

order models can be balanced by a predetermined threshold. The error bound A is de- 

fined as 

X Yin OF 
Np > 
1107 

M is then chosen such that A(M) < A* for a desired threshold A* [64]. Typically, M can 

be chosen rather small compared to N; while still keeping A close to zero (typically 

< 107). 

After computing the POD basis set, a reduced-order model is derived by projecting 
the PDAEs of the system onto the corresponding POD subspace. This means that we 
seek an approximation of the form 


À(M)=1 (1.4) 


M 
Z(E, t) = Zpop(&,0) = ) aj(t) i). (1.5) 


i=1 


To demonstrate how the Galerkin approach is applied to determine the coefficients 
a;(t), consider a PDE in the following form: 


cae f(z, a) (1.6) 


Using the POD basis functions as the weighted basis functions for the Galerkin projec- 
tion, we obtain the system 


da; 7 u M dg; Da 
ae ne OEY O JH dë, i=1...M, (1.7) 
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leading to a set of M ordinary differential equations (ODEs). If the spatially discretized 
system were directly solved with the method of lines, it would consist of Nz ODEs. 
Since M is normally much less than N;, POD can create a much smaller model that 
still maintains reasonable accuracy. 


1.2.2 Data-driven reduction 


Data-driven reduction methods have been successfully applied to truth models where 
projections and basis functions cannot be generated from the model equations, and 
only input/output responses are available from a black-box code. This black box may 
be sampled, and regression/interpolation approaches can be used to fit the sampled 
data. The resulting surrogate model replaces the truth model for simulation, optimiza- 
tion, or other analysis. There is considerable flexibility in the functional form and 
fitting methods used for surrogate construction, and this flexibility can be used to 
customize an approach suitable for a particular problem. Simpson etal. [63] provide 
a review of the field, which outlines several important steps and existing surrogate 
modeling frameworks. The main steps of surrogate model construction include experi- 
mental design, model selection, and model fitting. Several established methodologies 
suggest combinations of choices for each of these three steps. For example, response 
surface methodology, typically used in optimization settings, uses central compos- 
ite designs in combination with quadratic models constructed with least-squares re- 
gression. The central composite design helps determine curvature information for the 
quadratic models. A more complete description can be found in Myers and Mont- 
gomery [59]. In some ways, response surface methodology is a predecessor to the trust 
region-based methods that will be discussed in Section 1.3. Other surrogate model- 
ing approaches include Gaussian process regression (including kriging) and artificial 
neural networks. These methods often perform better with space-filling or sequential 
experimental designs [45, 36]. 

Recent work also examines the role of model complexity in surrogate modeling. 
When simpler functional forms are preferred, best-subset techniques combined with 
integer programming can be used to fit models [29, 72]. Moreover, recent develop- 
ments in machine learning have led to a wealth of new methods for model reduction 
[58, 65]. 

An example that demonstrates many concepts from data-driven model reduction 
may be found in [48]. That work proposes a model reduction method for distributed 
parameter systems based on principal component analysis and neural networks. The 
reduced model is designed to represent the spatially distributed states z of the sys- 
tem as functions of the inputs w, including boundary conditions, equipment param- 
eters, operating conditions, and input stream information. Similar to POD, this PCA 
approach seeks to represent the states in terms of a set of empirically determined ba- 
sis functions. First a set of snapshots is determined by running the truth model at 
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various input conditions W = {w,...u,, }, giving the snapshot set 


Z = {z(&,Wy),...,2(&, Wy) }, (1.8) 


where each snapshot z(č, w;) is a column vector representing the spatial profile at in- 
put point w;. There are n, snapshots and N; spatial discretization nodes. The basis 
functions are obtained in the same manner as discussed with equations (1.2), (1.3), 
and (1.4). After obtaining the reduced basis set ġ;(¥) (the principal components), the 
reduced model is expressed as 


M 
2(&,W) = Zpca(Ew) = © awo). (1.9) 
ial 


Whereas POD determines the coefficients a; through Galerkin projection of the truth 
model equations onto the basis set, the PCA-RM approach of [48] uses neural net- 
works. In other words, each function a;(w) is the result of training a neural network to 
capture the nonlinear relation between the input variables and the states represented 
with the principal components. This PCA-RM approach was applied to a CFD model of 
an entrained flow gasifier, embedded within an open-equation advanced power plant 
model [49]. The truth model was implemented in Fluent and takes around 20 CPU 
hours to solve. With both high computational cost and the use of commercial tools 
that may be difficult to use for custom analysis and simulations, this problem has both 
motivating features for the use of reduced models. There were three input variables for 
the truth model, including the water concentration in the slurry, oxygen to coal feed 
ratio, and the ratio of coal injected at the first feed stage. As described in [49], the 
resulting PCA-based RM had very good accuracy, as validated by leave-one-out cross- 
validation. 


1.3 Process optimization using reduced models 


Many process engineering applications on reduced modeling involve optimization for- 
mulations. One of the challenges in this field is the size of the problems created if the 
system is fully discretized before optimization. In addition, optimization routines are 
not easily customized to handle a particular problem as simulation. Here, a reduction 
in problem size can greatly speed solutions to enable real time application. 

However, despite significant effort in building reduced models, it is known that 
using an RM in optimization can lead to inaccurate answers. Small errors introduced 
by the RM approximation can propagate through to large errors in the optimum so- 
lution. This is worsened by the optimization’s tendency to exploit error to artificially 
improve the objective function, and hence optimization may terminate in regions of 
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poor RM accuracy. The RM can be refined sequentially during optimization, using in- 
formation from the truth model to improve inaccuracies. However, these iterative im- 
provements offer no convergence guarantees to the optimum of the high-fidelity opti- 
mization problem, even if it does converge at a point where the RM matches the truth 
model. 

The nonconvergence behavior can be observed through a toy problem in [15, 16], 
shown as follows: 


min F(x) = (xP + (x9 (1.10) 
s.t. (x) =x? - (x2 — (x)? -1= 0, 


where we denote the cubic function t(x) as the truth model. As shown in Figure 1.1, the 
problem (1.10) has a global minimum, (x*) = (0,1), f(x*) = 1, and a local maximum, 
(x*) = (-1,1), f(x") = 2. Now consider the corresponding RM-based problem given by 


min f(x) = (xy + (x)? (1.11) 
s.t. r(x) = x? -x® -b= 0, 

where we denote the linear function r(x) as the RM with an adjustable constant b. It 
is straightforward to show that the solution to (1.11) is given by x* = (—b/2, b/2), and 
f(x") = b’/2. Moreover, as shown by the steps in Figure 1.1(a), the RM-based algorithm 
proceeds at iteration k by choosing b, so that r(x;) = t(x;,) = 0. Then (1.11) is solved to 
obtain the next iterate x,,,;. However, this approach does not guarantee convergence 
to the optimum for the truth model. For instance, if we start from (0,1), the global 
minimum solution of (1.10), Figure 1.1(a) shows that the iterates x, actually move away 
from this solution and eventually converge to a nonoptimal point where t(x) = r(x), 
b = 2, and x = (-1,1). For this example, it can be seen that this point is actually a 
local maximum of (1.10). This behavior arises because optimality conditions rely on 
derivative information, not simple matching in function values. 

The most common approach to “safe” optimization with reduced models is to use 
a trust region method. Instead of approximating a black-box function over the entire 
domain of decision variables, a reduced model is constructed to locally approximate 
over this trust region. Assuming sufficient data are available, smaller domains can 
lead to lower absolute error in the reduced model and the choice of functional form 
becomes less critical with the restricted approximation. Trust region methods exploit 
this feature by adapting the trust region radius during the optimization process. 

Most trust region algorithms adopt the following outline. As a general example 
assume that the goal is to solve the following optimization problem: 


minh di) (1.12) 
s.t. g(x) <0. 
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(a) (b) 


Figure 1.1: Characteristics of toy example. (a) Convergence failure. (b) Convergence with trust region 
method. 


The initial point is denoted as x9. At each iteration, a trust region method will first 
construct an approximation of (1.12) that is valid on the trust region at iteration k. In 
other words, identify f and ê such that 


fo =f) and B(x) = g(x) forx € BX Ay), 


where B(x% Ap) = {x : |x — x;|| < Aj} is the trust region at iteration k. In classical trust 
region methods for nonlinear optimization, a quadratic approximation of the objec- 
tive function is constructed, while the constraint functions are linearized. However, 
alternative forms may be used and the characteristics of the optimization algorithm 
change depending on the type of approximation (type of RM) and the nature of accu- 
racy required for the approximation. 

The second step is to solve the trust region subproblem. This means that the re- 
duced models are used to optimize the function within the trust region, where suffi- 
cient accuracy is assumed. For our example problem (1.12), the trust region subprob- 
lem is 


min f(x) 
s.t. g(x) <0, (1.13) 
IX — xxl] < Ag. 


The trust region constraint helps prevent the algorithm from extrapolating into re- 
gions where the RM is not accurate, and provides a globalization mechanism to make 
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sure the algorithm converges. On the other hand, algorithms vary on how the trust re- 
gion subproblem is solved, and even the type of subproblem considered (constrained 
vs. unconstrained, convex or nonconvex). 

The final step is to evaluate the solution proposed by the trust region subproblem, 
denoted as x. Using recourse to the truth model, the solution can be evaluated in terms 
of accuracy. Depending on the improvementat x, the algorithm determines that either 
X41 = X, in which case we say that the step was accepted, or X,,, = X;,in which case we 
say that the step was rejected. The algorithm also determines the trust region radius 
for the following iteration Axı. There is significant freedom in algorithm design to 
handle this last step, with various ways to decide on when to accept or reject the step 
and how to update the trust region radius. 

Alexandrov et al. [3] applied the trust region concept to general reduced models 
in engineering. They considered the task of unconstrained minimization using arbi- 
trary approximation functions in place of the actual function. Reduced models are 
constructed so that the function and gradient values of the reduced model match those 
of the truth model at the center of the trust region. The trust region subproblem was 
the minimization of the reduced model subject to the trust region constraint. Stan- 
dard methods for unconstrained trust region methods were used to evaluate the step, 
including the ratio test [26]. Convergence was proved to the optimum with the truth 
model. These concepts were extended to classes of constrained optimization problems 
in the DAKOTA package [43]. Moreover, early work in [4, 39] demonstrated the use of 
POD reduced models with a trust region method. This algorithm was shown to be con- 
vergent under the assumption that the gradients of the POD model are sufficiently 
accurate, although this condition may be difficult to verify in practice. 

To illustrate the RM-based trust region approach, we revisit problem (1.10) but 
consider the NLP associated with the following RM: 


min f(x) = (xy + (x)? (1.14) 


a ee 


s.t. r(x)= x? 
where the RM has adjustable constants a and b. The corresponding trust region prob- 
lem is given by 


min (x® + s) + (a(x +s) + by) (1.15) 


s.t. [IS < Ak (1.16) 


and the progress of the trust region algorithm is sketched in Figure 1.1(b). Using Ay = 
0.8, the trust region algorithm converges to a tight tolerance after 20 iterations [16]. 
For the case where gradients are not available from the truth model, Conn etal. 
[27] discuss sufficient accuracy conditions on the reduced model to guarantee conver- 
gence. This condition, called the x-fully linear property, can be verified for data-driven 
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reduced models (e. g., polynomial interpolation). The x-fully linear property (see (1.20) 
below) dictates how the differences between reduced and truth models must scale di- 
rectly with the trust region radius. In this way, shrinking the trust region allows first- 
order optimality to be guaranteed in the limit. These derivative-free trust region ideas 
were extended by March and Wilcox [55] to consider the use of multifidelity models. 
In that study, the reduced model was a coarse discretization of the PDE system. Wild, 
Regis, and Shoemaker also use the framework of x-fully linear models to develop an al- 
gorithm with radial basis functions as RMs [70]. March and Wilcox [55] use constrained 
trust region subproblems with reduced models, with globalization managed with the 
use of a merit function. More recent extensions of these optimization strategies and 
applications for PDAE systems are reviewed in [61]. 

While the RM-based trust region methods guarantee convergence to the truth 
model-based optimization problem, the RM itself needs to be updated frequently, po- 
tentially for each trust region subproblem. To address this limitation, recent work on 
construction of RMs with embedded parameters (e. g., decision variables) appear to 
be particularly promising [11]. In particular, the idexempirical interpolation method 
(EIM) and the discrete EIM (DEIM) develop RMs that contain an interpolant for pa- 
rameter values [7, 25]. This allows fast updates of the RM as part of the optimization 
process, with much less evaluation of the truth models. 

Finally, for the optimization of multiscale chemical processes, Caballero and 
Grossmann [23] use kriging models to represent unit operations for process opti- 
mization and trust regions were used to shrink the domain of the kriging models, 
though convergence to local optima was not proved. Agarwal et al. [1] consider the 
optimization of periodic adsorption processes with POD-based reduced models. For 
this system they analyze and present convergence results in [2] for constrained sub- 
problems when derivatives of the truth models are known. For the related simulated 
moving bed (SMB) process with linear isotherms, Li etal. [52] develop and apply 
system-theoretic model order reduction methods to accelerate the computation of the 
cyclic steady states and optimize the SMB system. In a related study, these authors 
develop and demonstrate an efficient trust region method with surrogate (POD as well 
as coarse discretization) models to optimize the SMB process [51]. Biegler et al. [16] use 
a penalty function to solve inequality-constrained problems when the derivatives are 
unavailable and suggest stopping criteria based on the reduced model errors. More- 
over, Bremer etal. [20] apply a POD-DEIM method to a dynamic, two-dimensional 
reactor model for CO, methanation. In their dynamic studies they demonstrate that 
the resulting RM is accurate and accelerates the solution of the truth model by over 
an order of magnitude. 

More recently, reduced models have also been used in global optimization/global 
search using “gray-box” models [13, 18, 19]. In this broader class of problems, sim- 
plified models stand in for challenging or computationally expensive modeling ele- 
ments. However, dimensionality of these cases is restricted and asymptotic behavior 
is ignored because of tight budgets on function calls. 
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In the next section, we summarize our work on a trust region method that pro- 
vides rigorous convergence guarantees to the truth model-based optimum. By extend- 
ing concepts from classical optimization theory to RM-based optimization, an algo- 
rithm is developed to automatically manage RMs and prevent inaccurate solutions. 
Building off the x-fully linear theory for RMs from [27] and classical optimization the- 
ory, the proposed algorithm is extensible to a wide range of RM-based optimization 
problems for chemical processes. 


1.3.1 A trust region filter approach for RM-based optimization 


In order to address process optimization problems with heterogeneous models illus- 
trated in Figure 1.4, we consider a slight extension of Problem (1.12). Let t(w) represent 
a high-fidelity truth model, which will be approximated with reduced-order models 
r,(w). We also refer to the truth model y = t(w) as a black-box model, which is embed- 
ded within a larger optimization problem as follows: 


min fw, y,Z) 

s.t. h(w,y,Z) = 0, 
g(w,y,Z) < 0, 
y=t(w). 


(1.17) 


Here w € R” andz € R”, and the functions f, h, g are assumed to be twice differen- 
tiable on the domain R™*"*? and form the equation-oriented process model. We refer 
to these functions as the glass-box model, where accurate derivatives are assumed 
to be cheaply available, e. g., through the use of automatic differentiation. The high- 
fidelity truth model is shown as a map t(w) : R” — R taking input variables w; 
y € R” represents a lifted variable that represents the output of this black-box model. 
This lifting isolates the truth model from the glass-box model equations and allows its 
replacement with a (glass-box) reduced model. The remaining decision variables are 
represented by z. 

In chemical processes, the black-box y = t(w) often represents a complex PDE- 
based unit operation, while the remaining glass-box constraints h and g represent the 
rest of the process. This allows a model to be multiscale in that a detailed model of 
transport and reactions can be coupled with process-level equation-oriented models. 
For simplicity, we introduce an aggregated variable vector x? = [w7,y7,z7]. 

To allow for convergence of the trust region algorithm, we also assume that t(w) is 
twice continuously differentiable. In comparison to the generic formulation in (1.12), 
this form is chosen so that the only approximation task is the replacement of t(w) 
with a reduced model. The remaining functions, including objective function and con- 
straints, remain unaltered in the trust region subproblem (1.18). With t(w) replaced by 
a glass-box reduced model r,(w) and a trust region constraint added to confine x toa 
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domain where the reduced model is presumed to be accurate, the trust region radius 
A, and trust region center x, are then updated by the algorithm to guarantee conver- 
gence: 


min f(x) 

s.t. h(x) =0, 
g(x) <0, (1.18) 
y=r,(w), 
IX — xxl] < Ay. 


The trust region filter (TRF) method for solving optimization problems with em- 
bedded RMs is presented in [37]. This algorithm combines filter methods for con- 
strained NLP with model-based trust region methods developed in [27]. The TRF 
framework controls errors in the reduced model while balancing improvement in 
feasibility and optimality. The algorithm is based on the sequential quadratic pro- 
gramming filter method in Fletcher et al. [40] and its extension to inexact Jacobians 
by Walther and Biegler [68]. Borrowing concepts from multiobjective optimization, 
the filter examines trade-offs in constraint violation and objective function improve- 
ment. 

We now briefly outline the algorithm, discuss convergence, and demonstrate the 
ability to solve practical problems. At each iteration, the TRF method constructs or 
obtains an RM with some specification on desired accuracy. Then, the RM is checked 
for compatibility with the rest of the model. Even though problem (1.17) may be fea- 
sible, the trust region subproblem (1.18) may be infeasible due to inaccuracies of the 
RM r,(w). For the convergence proof, compatibility is enforced by a slightly stricter 
condition, and requires that a feasible point lie within the ball B(x,, A, min[1, KA), 
where x, and y are tuning parameters. After verifying compatibility, a criticality check 
is performed to determine whether the algorithm may be near convergence. 

Once the trust region subproblem (1.18) is solved, the solution, xg, is used to check 
the accuracy of the RM. The truth model is evaluated at the proposed solution, t(w,), 
and this value and the objective value f (x) determine whether to accept or reject the 
proposed step using the filter method [41]. For the purposes of RM-based optimization 
of form (1.17), constraint violations are equivalent to the inaccuracy of the RM. 

Depending on the accuracy condition on the reduced model, the algorithm can 
take several forms. Agarwal and Biegler [2] consider the case where the function and 
gradient values of the reduced model must agree with the truth model, i. e., t(w;,) = 
r,(w;,) and Vt(w;,) = Vrę(wķ). This requires accurate gradient information from the 
truth model. In fact, to construct these models for any RM, 7(w), one can define [3] 


ry (W) = F(w) + (t(wy) — F(W,)) + (VECW,) - Vi(w;,)) (w -= Wy) (1.19) 


i. e., through zero-order and first-order corrections, as shown in (1.27) and (1.28), re- 
spectively. 
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On the other hand, when the truth model gradients are unavailable, one can use 
the x-fully linear property instead, as developed in [37] and defined as follows. 

A model r;(w) is said to be x-fully linear on B(w,, Ax) for constants xç and Kg if, for 
all w € B(w;,, Ay), 


Irw) - t(w)| < Kp Ay and ||Vr,(w) - Vt(w)|| < K Ax. (1.20) 


The constants kp > 0 and x, > O can be derived from the particular form of 
r,,(w) and the data used to construct r;,(w) if applicable. We assume they are uniformly 
bounded, but exact values of these constants are not needed in the algorithm. 

This property requires that the accuracy of the RM needs to behave in a similar 
manner to a first-order Taylor expansion. This accuracy condition is easily verified 
when using interpolation models, such as polynomials [27] or radial basis functions 


[69]. Of course, if r,(w) is given by (1.19), the x-fully linear property applies directly. 


1.3.2 Summary of the trust region filter algorithm 


A detailed description of the TRF algorithm along with the convergence analysis and 

properties can be found in [37]. A summary of the key steps is listed below. 

1. Initialization: Choose the initial point x and trust region size Ag, as well as con- 
stants that guide the expansion and contraction of the trust region and the initial 
filter. Evaluate t(wọ) and then calculate 0(x0) = |l¥o — t(Wo)|| and set k = 0. 

2. RM construction: If Vt(w,) is available, calculate an RM r;,(w) using (1.19). Else, 
generate rą(w) that is x-fully linear on A,. 

3. Compatibility check: Solve a compatibility problem to determine if the trust region 
problem (1.18) is feasible. If so, go to Step 4. Otherwise, add (0%, fy) to the filter and 
go to Step 7. 

4. Criticality check: At x, compute a criticality measure for problem (1.18) without the 
trust region constraint, and check whether the Karush—Kuhn-Tucker (KKT) con- 
ditions hold within tolerance. If this condition holds and the RM was constructed 
from (1.19), then STOP. Because Vt(w;,) = Vr(w,) then, the KKT conditions of prob- 
lem (1.17) hold as well. If Vt(w;,) is unavailable and the criticality check holds, 
reassign A; := wA, and go to Step 2. Else, continue to Step 5. 

5. Trust region step: Solve the subproblem (1.18) to compute a step sx. 

Filter: Evaluate 6(x; + s) and determine whether it is sufficiently decreased from 
6(x;,). Follow the filter steps described in [37] to manage the trust region size, up- 
date the filter and accept x;,,, = x,+5;, or reject the step and set x,,, = Xg. Return to 
Step 2. (A detailed description of all of the steps in the filter procedure is presented 
in [37].) 

7. Restoration phase: As described in detail in [37], compute x;,;, Ay, and rg that 
is compatible to (1.18) and is acceptable to the filter. Set k = k + 1 and return to 
Step 2. 
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The global convergence analysis of the above algorithm requires standard assump- 
tions: x-fully linear RMs, smoothness of the underlying functions, and regularity of 
limit points for problems (1.18) and (1.17). The convergence proof is based on the TRF 
analysis of Fletcher et al. [40]. With slight modifications [37] it extends to (1.18) to show 
existence of a subsequence Xk, for which Xk > x", where x" is a first-order KKT point 
of (1.17). This global analysis also shows that feasibility and criticality measures limit 
to zero at x”. Full details of the convergence proof can be found in [37]. 


1.4 RM-based process optimization case studies 


This section presents two challenging case studies with complex truth models, the first 
with available truth model gradients, and the second without. These cases demon- 
strate the effectiveness of the TRF algorithm for RM-based process optimization. The 
first case study considers the optimization of a pressure swing adsorption process 
for carbon capture using POD reduced-order models, where we assume gradients 
are available from the truth model. The second case study considers optimization of 
an oxycombustion power plant, in which the boiler is modeled with a one-/three- 
dimensional hybrid zonal approach, leading to a PDE-based truth model, which is 
reduced using data-driven methods. Here the truth model does not provide gradient 
information. Additional studies that demonstrate trust region methods for RM-based 
optimization can be found in [16, 49, 38]. 


1.4.1 Pressure swing adsorption 


Pressure swing adsorption (PSA) technology is widely applied for separation and pu- 
rification of gas mixtures. By feeding gas through a bed of solid adsorbent, a stream 
enriched in the less strongly adsorbed components is produced. This normally oc- 
curs at high pressure to favor more adsorption. When pressure is reduced, equilib- 
rium is shifted and the adsorbed components will desorb, thus regenerating the solid 
adsorbent for reuse. The desorbed gas is enriched in the more strongly adsorbed com- 
ponents of the feed gas. By cycling through adsorption and desorption, the feed gas 
stream is separated into two purified streams. 

This cyclic process of adsorption and desorption driven by changing pressure 
lends the technology its name. PSA normally occurs in one or more packed beds. 
Because feed gas is only fed during the pressurization step and the outlet gas com- 
position changes for each step, it is common to have multiple beds in parallel to 
maintain continuous operation. Because PSA is often used in continuous processes, 
the beds are operated in cyclic steady state. For modeling purposes, cyclic steady-state 
operation requires imposition of a periodic boundary condition which enforces the 
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same state at the end of the cycle as the beginning. The conditions within each bed 
can be described by nonlinear PDAEs for complex fluid flow and nonlinear adsorp- 
tion isotherms. The solution of these PDAEs is governed by steep adsorption fronts, 
which require a large number of discretization points. In addition to the large set of 
discretized equations from the PDAE, the optimization model for PSA includes cyclic 
steady-state conditions and stringent purity constraints. The scale of this optimization 
problem, whether considering design or operation, presents a significant challenge to 
state-of-the-art nonlinear programming algorithms. This motivates the use of model 
reduction to reduce the size of this system. 

A two-bed four-step isothermal PSA process is shown in Figure 1.2. The feed mix- 
ture is 85% N, and 15 % CO). The four steps of operation are pressurization, adsorp- 
tion, depressurization (counter-current), and light reflux (or desorption). The two beds 
are operated as follows. First, bed 1 is pressurized with feed gas while bed 2 is de- 
pressurized, producing a stream rich in the strongly adsorbed component. Next, high- 
pressure feed gas is continually added to bed 1, and the heavy component continues to 
adsorb, producing a product rich in the weakly adsorbed component (light product). 
A fraction of the light product gas is fed to bed 2 at low pressure to purge and further 
desorb the accumulated heavy adsorbate. This is called the light reflux step. Next, the 
two beds interchange roles, with bed 1 first depressurizing and then receiving light 
reflux. Thus, with four steps, the system returns to the original state. 


(Step 3) (Step 4) (Step 1) (Step 2) 
Counter-current Light reflux Feed Feed 
depressurization (or Desorption) pressurization (or Adsorption) 
Heavy Heavy Feed Feed 
product product 

Bed 2 Bed 2 Bed 2 Bed 2 
A A 
Purge Outlet} Light 
> product 
Light product p m> 
Outlet urge 
y 
Bed 1 Bed 1 Bed 1 Bed 1 
[Feed | Feed | Heavy J Heavy 
product product 
Feed Feed Counter-current Light reflux 
pressurization (or Adsorption) depressurization (or Desorption) 
(Step 1) (Step 2) (Step 3) (Step 4) 


Figure 1.2: A two-bed four-step PSA cycle. 
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The mathematical model for the PSA process is presented in Table 1.1. The model as- 
sumes all gases to be ideal and radial variations in concentration are neglected in 
both gas and solid phases. In addition, the process is assumed to be isothermal with 
negligible pressure drop along the bed. Adsorption is modeled with the dual-site Lang- 
muir isotherm and the linear driving force (LDF) expression. Zeolite 13X is chosen as 
the adsorbent. Model parameters can be found in [46]. The four steps of the process 
are enforced using boundary conditions on the feed flow rate and composition. This 
serves as the truth model for which exact gradients are available for construction of 
the reduced model. 


Table 1.1: Model equations for isothermal PSA. 


Component mass balance 


dy; O(vy;) RT 0g; . 

b Ft + OE + (1 - €))P, Ae =0 i=CO, (1.21) 
Overall mass balance 

ov RT 0g; 

Oh gee ps te 1.22 
og + P ( Ep)Ps a ot ( ) 
LDF equation 

ðq; P ; 

Ht kla -a) i=12 (1.23) 


Dual-site Langmuir isotherm 
s  FibyyP F Dib aiy;P 


g = i = CO,, N (1.24) 
j j 

Cyclic steady state 

z(to) = 2(teycle) Z:Vpq, i= CO, N, (1.25) 


k; lumped mass transfer coefficient for i-th component (sec?) 

P total bed pressure (kPa) 

qi solid-phase concentration of i-th component (gmol kg’) 

q; equilibrium solid-phase concentration of i-th component (gmol kg?) 
R universal gas constant (J gmol! K~*) 

T gas-phase temperature in the bed (K) 

v gas superficial velocity (msec~*) 


y; mole fraction of i-th component 


Greek letters 
€» bulk void fraction 


ps adsorbent density (kg m~?) 


18 —— J.P. Eason and L.T. Biegler 


The model will be used in an optimization study to maximize CO, recovery subject to 
purity constraints. The key decision variables include the high pressure P, at which 
the adsorption step takes place, the low pressure P; for the depressurization and des- 
orption steps, the duration ¢, for pressurization and depressurization, the duration 
t, for adsorption and desorption, and the feed velocity during adsorption u,. These 
variables are aggregated into the vector w = [Pp Pi tp» ta» Ual, which represents the 
inputs to the PSA model. Given particular values for w, the truth model can be simu- 
lated to cyclic steady state. Starting values for these variables are shown in Table 1.2. 
However, to make the optimization problem more tractable, the PSA truth model is 
replaced with reduced-order models. 


Table 1.2: Initial guess for optimization problem (1.26). 


Decision variable Guessed value 
Adsorption pressure (P;,) 150 kPa 
Desorption pressure (P)) 50kPa 
Pressurization step time (tp) 50sec 
Adsorption step time (t,) 150sec 
Adsorption feed flow (u,) 20 cm/sec 


Table 1.3: Comparison of truth model and RM based on the performance variables. 


Performance variables Truth model RM 
N, purity 92.57 % 92.51% 
N3 recovery 80.21% 80.71% 
CO, purity 37.76% 38.29% 
CO, recovery 66.27 % 67.44% 


The RM for PSA is formed using POD. In fact, the reduced model is rebuilt several times 
during optimization as the decision variables change. We represent the sequence of 
decision variables as {wx}, k = 1,2,.... Each time a POD model is built, the truth model 
is first simulated at the particular point w, to gather a set of snapshots. To simulate the 
truth model, the spatial dimensions were discretized with 50 finite volumes and the 
resulting system of 1400 DAEs was simulated using MATLAB’s ode15s. Then, the size of 
the POD basis is determined using a cutoff parameter A* = 0.05. The Galerkin projec- 
tion is applied to obtain the RM, with equations shown in [2]. The RM has only 70 DAEs, 
which is a reduction of a factor of over 20 from the truth model. The RM equations are 
then discretized in time using Radau collocation on finite elements and the resulting 
equations are solved using IPOPT in AMPL. Table 1.3 compares the performance of the 
truth model and the RM for typical inputs. Further information on the model reduc- 
tion, implementation of the TRF method, and its performance can be found in [2]. 
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The RMs are used to maximize CO, recovery subject to a constraint on CO, purity. 
These variables are determined by a time average of the product stream composi- 
tions over the full-cycle time horizon. The optimization formulation is summarized as 
follows: 


max CO, recovery (time averaged) 
s.t. CO, purity = 0.5 (time averaged), 
101.32 kPa < P, < 300kPa, 
40 kPa < P; < 101.32 kPa, 
35sec < t, < 150sec, 
50sec < t, < 400 sec, 
10 cm/sec < u, < 30 cm/sec, 
PDAEs for PSA. 


(1.26) 


The bound on CO, purity is set at 50 %, which is determined by what is realistically at- 
tainable with a two-bed four-step cycle. Also, note that the pressure P, is allowed to lie 
in the vacuum range, which aids desorption. To improve computational performance, 
the PDAE system in (1.26) is replaced with a sequence of RMs. The RMs are used in a 
trust region method to find the optimum of problem (1.26) considering the truth model. 

The predicted CO, purity and recovery from the RM will often be different than 
those from the truth model. To ensure consistency and ultimately drive convergence 
to the truth model optimum, correction terms are added to the RM, as proposed by 
[3, 43]. The trust region method for the PSA problem used both zero- and first-order 
additive corrections (ZOC and FOC). The zero-order correction forms r,(w) that agrees 
with the truth model at the trust region center wg: 


Te (W) := rpop(w) + t(W;.) — Tpop (Wx) (1.27) 
so that the zero-order corrected outputs used for optimization are the same as pre- 
dicted by the RM with an additive correction factor. 

The first-order additive correction from (1.19) is given by: 


ro"(w) := Tpop(W) + (t(W,) - Tpop(Wx)) + (Vt(w;) - Vrpop(W,)). (w - w“), (1.28) 


so that the corrected outputs ne used for optimization agree with the truth model in 


function values and gradients at the design point wx. In addition to the simple Taylor 
expansion, the underlying POD RM provides a physics-based functional form to model 
how the outputs change further away from the design point w,. 

The zero- and first-order corrected RMs are used in a two-stage algorithm. The 
analysis in [2] shows that the trust region filter approach with first-order consistent 
RMs will converge to the optimum of (1.26). The POD models plus first-order correc- 
tion satisfy this first-order consistency condition, but are relatively expensive to build 
because of the need to estimate gradient information. Therefore, the approach used 
to solve this problem will first attempt optimization using the zero-order correction 
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models. The zero-order correction comes at no additional cost since the point w, is 
already evaluated with the truth model to gather snapshots to build POD models. The 
TRF approach is applied with zero-order correction models until no more progress is 
made towards the optimum. Then, the algorithm is reinitialized and first-order cor- 
rected models are used to guarantee convergence to a stationary point. The sensitivity 
information for first-order corrections is obtained using finite difference perturbations 
of the inputs w and simulating the truth model. 

The two-phase TRF algorithm successfully solves problem (1.26) with 51iterations, 
of which the first 35 use the zero-order correction. At each iteration, the discretized 
RM-based optimization problem is a nonlinear program with 52,247 variables. The 
CO, purity and recovery are plotted over the course of the iterations, as shown in Fig- 
ure 1.3. As seen in (24) CO, purity is constrained to be above 50 %, and this constraint 
is clearly active at most steps of the algorithm. However, maximization of CO, recovery 
is more difficult for the algorithm. Progress is slow, but after the zero-order correction 
approach terminates the solution is still suboptimal. In the decision variable space, 
this final increase in recovery in the first-order correction phase is largely achieved by 
moving the pressurization step time from near its upper bound to being active at its 
lower bound. This indicates that the POD-based RMs were not accurately capturing 
derivative information, which led to suboptimal results. The final termination point 
was further validated to confirm that it is a local maximum. 


100 T r r 1 
a OREO 
90F J 
~ CO, recovery „° i 
= 80F Wi 4 
= jt christensen e step accepted 
S cas * step rejected 
SC 
Cc 
i} 
æ 607 4 
5 
iz CO, purity 
Q sof °°eesooososoooosooo E E EE LE A TE ARTA AN 
40F° J 
30 1 L 1 Í fi 
0 10 20 30 40 50 


Iteration Number (k) 


Figure 1.3: CO, purity and recovery for all the iterations of the filter-based algorithm. 


The full results of this case study may be found in [2]. That work also compares the 
filter-based approach to a more conventional exact penalty approach. The benefit of 
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the filter mechanism is clearly shown by its flexibility to accept larger steps. In contrast 
to the TRF approach, the exact penalty method takes 92 iterations, each using the first- 
order correction. The filter’s flexibility to accept steps that may reduce feasibility give 
it an advantage of faster convergence. 

The PSA case study demonstrates the limitations of RM-based optimization while 
also demonstrating a solution in the form of first-order corrected reduced models. 
However, the expense of obtaining derivative estimates to build the first-order correc- 
tion counteracts the benefits gained from the model reduction. As a result, the two- 
stage approach applies first-order corrections as necessary to ensure convergence of 
RM-based optimization. 


1.4.2 Advanced power plant optimization 


The second case study is also an application of CO, capture. Unlike the previous study, 
we now deal with an entire flowsheet with both glass-box and black-box models. While 
first and second derivatives are calculated cheaply from glass-box models, gradient 
information is not available from the black-box truth models. These need to be ap- 
proximated by equation-oriented reduced models. 

This case study deals with the optimization of the oxycombustion process, one of 
several approaches for carbon capture from power generation processes. In oxycom- 
bustion, a mixture of purified oxygen and recycle carbon dioxide is used to combust 
fuel in a boiler. This results in a flue gas that is primarily water and CO). The water is 
easily separated and the task of purifying the CO, is greatly simplified because most 
nitrogen was removed before combustion. Compared to conventional air-fired power 
plants, the oxycombustion process has several key differences. The introduction 
of new separation tasks, namely, air separation and CO, compression/purification, 
lower the overall power output of the plant due to the energy required to run them. 
In addition, the recycle loop further couples the temperature and composition of 
pre- and postcombustion gas streams. Finally, the oxy-fired boiler behaves differently 
than air-fired boilers, and it is necessary to consider the effect of these changes on the 
rest of the power plant design. To rigorously manage the interactions between these 
subsystems and reduce the cost of carbon capture, a comprehensive optimization 
approach is proposed. 

In order to optimize the oxycombustion process, the first step is to consider the 
level of model fidelity required. For process optimization, steady-state models are usu- 
ally sufficient. At the process level, heat exchanger, pumps, compressors, and turbines 
are modeled as AEs. A framework for building these models as well as application to 
the oxycombustion separation systems was presented in [33, 32]. However, modeling 
the boiler presents special challenges. A full-scale CFD simulation including reactions 
and transport behavior in three dimensions can take several weeks to solve. Instead, 
we use a hybrid one-/three-dimensional zonal boiler model as described in [54] as the 
truth model. This boiler model uses a series of nine vertical zones to model reactions 
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and particle/gas flow, while integrating the radiation PDE in three dimensions with 
the discrete ordinate method on 21,888 finite elements. The resulting truth model can 
converge in about 1 CPU minute on a desktop computer. The model is custom built 
in C++ with specialized methods to help guarantee robustness when converging the 
simulation. As discussed in Section 1.2, the requirement for specialized simulation 
methods for the truth model suggests the use of data-driven model reduction. The hy- 
brid one-/three-dimensional boiler model is approximated with RMs and used to solve 
a plant-wide optimization problem with the TRF method. 

For the purposes of the optimization problem, the boiler model can be viewed 
as a function from R” — R”, where the input and output variables are chosen to 
capture the interactions of the boiler with the rest of the power plant. These inputs 
and output variables are given below (corresponding to the w and y variables in the 
glass-box/black-box formulation (1.17)): 


Boiler inputs: 

primary air temperature, 

secondary/over-fired air temperature, 

average temperature of boiling water inside water wall tubes, 

average secondary superheater steam temperature, 

primary air component flow rates (O2, N2, H2, CO, CO3, H30, SO, H-S, CH,, Ar), 
secondary air component flow rates (same components as primary air, but differ- 
ent compositions), 

7. over-fired air total flow rate (same composition as secondary air). 


AY PWNS 


Boiler outputs: 

1. boiler enclosure water wall heat duty, 

2. secondary superheater heat duty, 

3. flue gas component flow rates (same components as primary air), 
4. flue gas temperature. 


The primary air is the gas stream into the boiler that carries the pulverized coal parti- 
cles, whereas secondary and over-fired air streams are added into the boiler directly to 
aid in combustion. The temperature and composition of these streams have a strong 
impact on the combustion behavior in the boiler. In general, higher heat transfer 
through the boiler enclosure wall improves performance, although the flue gas tem- 
perature must remain bounded for material considerations. The compositions flowing 
into the boiler are indirectly coupled with the composition leaving the boiler through 
recycle, and the heat transfer behavior is also indirectly coupled with the average 
temperature of water in the water wall tubes through the steam cycle. The use of a 
rigorous boiler model helps capture these interactions accurately. 

Figure 1.4 shows the general configuration of the steam and gas sides of the steam 
cycle. On the steam side, high-pressure water enters at the bottom of the boiler and 
steam exits the boiler after the secondary superheater. The turbines are divided into 
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Figure 1.4: The water/steam and gas sides of the steam cycle. Heat exchangers are modeled as 
halves, where red units receive heat and blue units provide heat. 


high-, intermediate-, and low-pressure sections (HP, IP, and LP, respectively). A super- 
structure of potential steam extraction sites between stage groups is considered as part 
of the optimization. The extracted steam may be sent to the boiler feedwater heaters. 
Heat exchangers are modeled with heat exchanger halves, specified either as a heater 
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(red) or cooler (blue). The heating and cooling duties are matched for these heat ex- 
changer halves in order to form heat exchangers in the power plant, including the 
primary superheater, reheaters, and the economizer. The remaining heat exchanger 
halves are matched through the formulation of a pinch-based heat integration model. 
This heat integration model is developed using the Duran—Grossmann formulation 
[34]; see [33, 74] for details on the implementation in this equation-oriented flowsheet 
framework. On the gas side, the purified oxygen stream is split before being mixed with 
recycled flue gas. The split allows primary and secondary air streams to have different 
compositions. After combustion, the flue gas is split to two series of heat exchang- 
ers, and then sent to pollution controls. The direct contact cooler/polishing scrubber 
(DCCP) is used to remove much of the water from the flue gas. The ratio of flue gas 
sent for water cooling is also a key decision variable due to the role of water vapor in 
the radiation behavior in the boiler. Then, a fraction of the flue gas is sent for com- 
pression and purification while the rest is recycled to the boiler. Primary air flow rate 
is bounded below to ensure at least a 2:1 gas-to-coal ratio to ensure that the gas can 
carry the coal. For safety reasons, the primary air stream also has upper bounds on 
temperature and O, mole fraction, of 400 K and 35 %, respectively. 

The TRF algorithm was used to maximize the thermal efficiency of a double reheat 
oxy-fired steam cycle. The optimization problem is summarized in (1.29). Both design 
and operational decisions could be modified for the boiler, but we consider the case 
where boiler geometry and burner configuration match an existing utility boiler (Paci- 
ficorp’s Hunter 3 unit). 

The objective of this study to maximize thermal efficiency with a fixed coal feed 
flow rate, with a small penalty for utility consumption: 


max thermal efficiency + p,,Qy 


2 Wiurbine = x Woump 2 Win Wepu Wasu 
thermal input rate 


s.t. thermal efficiency = 


> 


fixed thermal (fuel) input rate, 
steam turbine model, 
pump and fan models, 
Duran—Grossmann pinch-location heat integration equations, Me?) 
correlation-based fuel gas thermodynamics model, 
steam thermodynamics, 
hybrid boiler model, 
correlation model ASU, 
correlation model CPU, 
where p,, is a small penalty term for cooling water usage. The air separation unit (ASU) 


and carbon dioxide processing unit (CPU) are modeled with correlations derived from 
[33, 30, 31]. 
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The truth boiler model is replaced with a sequence of reduced models rą(w). In 
contrast to the PSA study, this case study uses an improved version of the TRF algo- 
rithm that does not require first-order consistency for convergence. Instead, the RMs 
must satisfy the x-fully linear property (1.20). Common choices such as interpolation 
or regression apply under mild assumptions (e. g., uniformly bounded second deriva- 
tives of r,(w)). This condition provides great flexibility for a wide variety of reduced 
modeling approaches. For the boiler model, simple polynomial interpolation models 
constructed with well-poised sample sets provided good performance. These RMs are 
updated automatically, with recourse to the truth models, by the TRF method. 

Optimization problem (1.29) was solved for two different scenarios. In Case A, the 
oxygen purity supplied by the ASU was fixed at 97 mol%, which means that the power 
requirement of the ASU is only dependent on the flow rate of oxygen supplied. In Case 
B, the oxygen purity was allowed to vary between 90 mol% and 98 mol%. This allows 
the optimizer to trade off the pre- and postcombustion separation tasks, while simulta- 
neously considering the interactions with the detailed kinetics and radiation behavior 
in the boiler. In the lower oxygen environment, gasification reactions are more favored 
and the emissivity of the gas mixture changes. The optimization results for both cases 
are given in Table 1.4. In Case A, the optimum design found by the TRF algorithm is 
an oxy-fired power plant with a net power output of 437.4 MW and a net efficiency of 
33.0 %. In Case B, the optimum solution has a net power output of 440.4 MW and an 
efficiency of 33.2%.’ Interestingly, in Case B the oxygen composition is pushed to its 
lower bound of 90 mol%. In both scenarios the optimizer pushes the steam tempera- 
tures leaving the secondary superheater and reheaters to their upper bounds of 835 K 
and 867K, respectively, as expected when maximizing thermal efficiency. Similarly, 
the lower bound of 0.068 bar for the condenser operating pressures is active. The op- 
timizer also pushes the temperature and oxygen content of the primary flue gas recycle 
streams (S408 in Figure 1.4) to their upper bounds of 400 K and 35 mol%. Another in- 
teresting conclusion lies in the recycle distribution of the flue gas. The temperature 
and composition of the flue gas (influenced by drying) has complex interactions with 
the detailed boiler model. By optimizing using the reduced boiler model, these inter- 
actions can be considered and additional efficiencies are identified. Additional infor- 
mation on the implementation and performance of the TRF algorithm for the oxycom- 
bustion case study can be found in [37, 35, 54]. 


1.5 Conclusions 


Reduced models have been used in many domains in chemical engineering to provide 
computationally tractable simulation and optimization. Most model reduction tech- 
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Table 1.4: Power plant optimization results. Case A: fixed oxygen purity. Case B: variable oxygen 
purity. 


CaseA Case B 
Work from turbines (MW) 568.2 570.3 
HP 94.5 94.5 
IP 267.3 267.7 
LP 206.4 208.1 
Pumping work (MW) 12.4 12.4 
Fan work (MW) 3.6 3.7 
Heat from boiler (MW) 520.6 531.3 
Boiler walls 446.4 457.3 
Secondary superheater 74.2 74 
Heat from flge Gas (MW) 659.2 653.0 
Primary superheater 201.0 220.5 
Reheater (HX201) 168.6 168.7 
Reheater (HX202) 146.6 148.4 
Economizer 143.0 115.4 
Heat rejected (MW) 620.9 623.3 
Fuel heat rate (MW) 1325.5 1325.5 
ASU power (MW) 71.2 65.6 
CPU power (MW) 43.6 48.2 
Net power (MW) 437.4 440.4 
Thermal efficiency (HHV) 33.0% 33.2% 
Flue gas recycle distribution 
Bypasses DCCP, to secondary rec. 29.7 % 31.3% 
To CPU after DCCP 28.0% 29.9 % 
To primary recycle after DCCP 23.6 % 25.3 % 
To secondary recycle after DCCP 18.7 % 13.5% 


niques can be categorized as model-based or data-driven. Model-based methods can 
exploit known information about the structure of the truth model to build reduced 
models, whereas data-driven methods are often suitable when specialized software is 
used for the truth model. 

When reduced models are used in optimization, the accuracy becomes an even 
greater concern. Optimal solutions are characterized by gradient information, which 
adds more demands on the reduced model accuracy. Trust region methods provide 
a systematic approach to manage the accuracy of reduced models in optimization. 
Through adaptive updating of reduced models, convergence can be guaranteed to so- 
lutions of the truth model-based problem. However, accuracy conditions must be en- 
forced on the reduced models. While first-order consistency is straightforward to en- 
force, this may be computationally demanding in practice. Instead, the framework of 
x-fully linear models provides more flexibility. 
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Our optimization strategy is agnostic to the type of reduced model as long as the 
x-fully linear property holds. The verification that an RM is x-fully linear is easy when 
a data-driven RM is used. On the other hand, if truth model gradients are not avail- 
able, it is not clear that the x-fully linear framework can be used for model-based re- 
ductions. Thus, while model reduction is already used in many domains of chemical 
engineering, challenges remain in building (certifiably) accurate and efficient RMs. 

Future work will develop more efficient TRF methods. Recent advances of the TRF 
algorithm includes the addition of a sampling region, which ensures the accuracy of 
the RM, while the surrounding trust region globalizes the TRF algorithm, and need 
not shrink to zero upon convergence. As developed, analyzed, and demonstrated in 
[38], this TRF enhancement led to a 40 % decrease in computational effort. Further 
research will also be devoted to tailored algorithms that exploit the solution of multi- 
ple RMs within the process flowsheet as well as specialized decompositions for truth 
models. 
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2 Model order reduction in mechanical 
engineering 


Abstract: This chapter describes several “success stories” of model order reduction 
(MOR) in mechanical engineering. First, the reader will be given an overview of spe- 
cific model representations, MOR requirements, and reduction techniques relevant in 
the different fields of mechanical engineering. Then, four applications are presented: 
the reduction of a thermo-mechanical machining tool, the reduction of models of a 
car body and driver’s seat, the reduction of an elastic crankshaft, and the reduction 
of a leaf spring model. 


Keywords: Model order reduction, mechanical engineering, second-order systems, 
projection 
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2.1 Introduction 


In the past 50 years, the number of applications of model reduction in mechanical en- 
gineering has increased massively, while the first applications in structural mechan- 
ics and acoustics ([94, 55, 56, 24], see also [18]) go back even further in time. Modal 
methods can be applied either to second-order models (common in multibody systems 
[48, 24, 1, 33, 84]) or to state-space representations. Corresponding methods were in- 
tensively developed from the 1960s onwards ([25, 73, 23, 20] and Chapter 4 of Volume 1 
of Model order reduction) and are applied in other areas of engineering as well, for in- 
stance in the reduction of power systems [64] and for the purpose of control design, 
e.g., [71, 66, 58, 7]. With the advent of balanced truncation and of Krylov subspace 
methods ([78, 44], [42, 45], overviews in [4, 8, 10], and Chapters 2 and 3 of Volume 1 of 
Model order reduction) the approximation quality and the applicability to high- and 
very high-order linear systems improved significantly and opened numerous fields 
of applications. Proper orthogonal decomposition (POD) methods based on snapshots 
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[63, 70, 60] and hyper-reduction techniques [22, 30, 31] were developed for the reduc- 
tion of nonlinear models. 

Today, the successful applications of model order reduction (MOR) in mechanical 
engineering deal with different system and problem classes from different physical 
domains, like: 

— structural and multibody dynamics, modeled by linear or nonlinear differential 

equations [18, 17, 21, 67, 69, 35, 34, 53, 3, 31, 91, 90, 5, 104, 19, 49, 47, 95, 110]; 

-— fluid dynamics, including fluid-structure interaction and aerodynamics [93, 26, 

89, 88, 83, 2, 31]; 

—  thermo-mechanical, thermo-fluid, thermo-acoustic, and thermo-electrical systems 

[68, 12, 14, 11, 97, 75, 54, 46]. 


An overview on coupled problems is [15]. It should be emphasized that this list of ref- 

erences is incomplete. In view of the huge number of publications on MOR applica- 

tions, it seems rather impossible to give a complete overview. Besides the many sci- 
entific publications, there exist several collections of benchmark problems, like the 

“Oberwolfach Benchmark Collection,” the “Niconet Benchmark Collection,” and the 

“MOR-Wiki.” These collections include system models from different domains and can 

serve for the validation process. 

In Sections 2.2 to 2.5 of this chapter, four successful applications are presented as 
examples of MOR of industrial problems: 

— The reduction of a coupled thermo-mechanical machining tool model applies 
Krylov subspace-based reduction to a first- or second-order formulation of the 
linear model. 

—  Thereduction of industrial models of a car body and driver’s seat applies a Craig- 
Bampton method and a Gramian-based method to second-order models of very 
high order. 

— The error-controlled reduction of an elastic crankshaft applies a combination of 
methods to a second-order model. 

—  Thereduction ofa leaf spring model applies simulation-free projection and hyper- 
reduction to a nonlinear second-order model. 


In the following, an overview of common representations for mechanical systems, 
modeling aspects, and basic properties will be given. 

Typically, mechanical models originate either from direct discrete modeling, spa- 
tial discretization of partial differential equations, system identification, or a combi- 
nation of those. 

Direct discrete modeling uses discrete, linear, or nonlinear mass, damper, and 
spring elements to build up a model. This process normally leads to small- or medium- 
sized models. 

Most models originate from spatial discretization of partial differential equations 
describing the laws of physics. For solids, a continuum mechanics approach — equi- 
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librium of forces, kinematic equations and the constitutive equations — describe the 
physics. Well-established methods like the finite element method (FEM), the finite dif- 
ference method, the finite volume method, the boundary element method, and oth- 
ers are applied for spatial discretization. All procedures are based on the method of 
weighted residuals and differ only in the specific choices of test and weighting func- 
tions. The classical displacement-based FEM, for example, uses first- or higher-order 
polynomial test and weighting functions from identical function spaces, i. e., a Ritz— 
Galerkin projection from the continuous to the discrete space is performed. This typ- 
ically leads to very large models comprising thousands up to millions of degrees of 
freedom. 

Another way of modeling mechanical systems is a data-driven identification of 
their dynamics. This typically leads to rather small models. 

All models have in common that the equilibrium of inertia forces, Mq(t), damping 
and internal restoring forces, fW, q(t)), and external forces, F(t), determines the 
basic dynamics of mechanical systems: 


Mätt) + flata) =F), 40) = dao 40) = Ao, (2.1) 


with (generalized) displacements q(t) € R, mass matrix M € R", nonlinear damp- 
ing and internal forces f : RY x RN — RY, and external forces or loadings F(t) € RY. 

External forces can also be considered explicitly as a space- and a time-dependent 
part, F(t) = BF(t): 


Mätt) +f (q(t), 4() = BF(t), q(0) = qo» 4(0) = ġo. (2.2) 


This allows for a system-theoretic point of view where the input-output behavior is 
of importance: The input matrix B € RY? contains weights and allocations to the 
degrees of freedom of the time-dependent forces, i.e., the input signals F(t) € R? 
(p < N). The corresponding system outputs are given by equation (2.5). 

Modeling of the dominating damping mechanisms is mostly not straightforward. 
Therefore, one frequently gets by with assuming simple linear viscous damping Dq(t). 
Additionally excluding gyroscopic effects allows for writing f(q(t), q(t)) = Dq(t) + 
f(q(t)), such that 


Mä(t) + Dq(t) + f(q(t)) = BF(t), (0) = qo; (0) = ġo, (2.3) 


with damping matrix D € R“*" and nonlinear internal forces f : RN > RN. 


For sufficiently small displacements around an equilibrium position, used as ini- 
tial configuration, only the linear part of the internal restoring forces, f(q(t)) = Kq(t), 
has to be considered. This leads to the well-known linear second-order representation 


Mq(t) + Dq(t) + Kq(t) = BF(t), q(0) = qo, (0) = Qo, (2.4) 


with stiffness matrix K € RY. 
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Typically, displacements, velocities, or linear combinations, e. g., stresses, at spe- 
cific nodes constitute the system outputs: 


y(t) = Cgq(t) + C,q(t), (2.5) 


with output matrices C4 € R?” and C gE R?” considering displacements and veloc- 
ities, respectively. 
The transfer behavior — inputs to outputs — is commonly represented as 


G(s) = (C4 + SC4)(S°M + sD + K)`™B (2.6) 
in the frequency domain, such that 
Y(s) = G(s)F(s), (2.7) 


where Y(s) and F(s) are the Laplace-transformed outputs y(t) and inputs F(t), respec- 
tively. 

While second-order representations are common in mechanics, state-space repre- 
sentations are often used in systems and control theory. Different implicit state-space 
representations exist, possessing desired properties for their system matrices. Here 
only, except for basis transformations, unique explicit representations will be given, 
while implicit ones are to be favored when, e. g., preservation of the matrix sparsity 
pattern is desired. The general nonlinear system representation (2.2) can be reformu- 


lated as 
E Leite) helo B-E e 


and the linear system representation (2.4) as 


Gol a an] gol lace)? my acon = [gel nr 


together with the output equation (2.5) rewritten as 


P 
4t] 

Systems involving coupling of either different components, i. e., multibody sys- 
tems, or of different physical domains, i. e., multiphysics systems, can show mass ma- 
trices M in second-order representations (or left-hand side matrices E in implicit state- 
space representations Ex(t) = Ax(t) + Bu(t), y(t) = Cx(t)) with rank deficiencies due 
to the algebraic interface equations. Thus, the system dynamics are described by dif- 
ferential algebraic equations (DAEs). An example is given in Section 2.2. 

Mass (M) and stiffness (K) matrices are symmetric and positive (semi-)definite for 
typical mechanical systems with appropriate boundary conditions suppressing rigid 
body modes. Implying those matrix properties directly results in the following system 
properties: 


y)=[C, C4] | (2.10) 
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—  Passivity: A mechanical system is passive for collocated inputs and velocity only 
outputs, i. e., B = C}, C, = O [109]. 

— Stability: A mechanical system is always Lyapunov stable. It is also asymptotically 
stable for positive definite D [59]. 


Commonly, linear damping is realized via modal damping. A simple and popular 
choice is the special case of proportional or Rayleigh damping, where D = aM + BK 
with a,ß = O, i.e., the damping matrix (D) is computed as a linear combination of 
mass (M) and stiffness (K) matrices [41]. With this, D is symmetric and positive defi- 
nite, the same as M and K. The conjugate complex eigenvalue pairs s;;,, = 0 + iw of 
undamped systems are all located on the imaginary axis. A mass proportional part, 
i.e., a > 0, shifts all eigenvalues with equal amount to the left, Ao = const., while a 
stiffness proportional part, i. e., $ > 0, shifts all eigenvalues to the left with an amount 
proportional to the frequency of the eigenvalue squared, Ao œ w°. 

In order to reduce the computational effort involved in numerically solving (2.4), 
a reduced-order model (ROM) that accurately approximates the behavior of the orig- 
inal full-order model (FOM) is aimed. This is usually achieved by projective MOR. To 
this end, the full displacements q(t) € R” are first approximated by a linear combi- 
nation of reduced displacements q,(t) € R” via the ansatz q(t) = V q,(t) + e(t), where 
Ve R” andn < N. Inserting this ansatz in (2.4), (2.5) yields an overdetermined 
system with the residual e(t) € RY, 


MV å (t)+DV å (t) +KV q(t) = BF(t) + €(t), 


(2.11) 
y,(t) = C4 Vq,(t) + C} V q,(t). 
To obtain a square system, we premultiply (2.11) by WT € R™, 
M, q.(t) F D, åd; (t) a K, q(t) = B, F(t), q,(0) = qro q,(0) = dro 212) 


y(t) = Cy, 4, (t) + Cy, q,(t); 


thus enforcing the Petrov-Galerkin condition W' c(t) = 0, where the residual e(t) 
vanishes. The reduced matrices are given by {M,,D,,K,} = Wim, D,K}V, B, = 
W'B, Ca, = C,V, and C}, = C;,V, and the initial conditions are {q,(0),q,(0)} = 
(wT V) Wq, qo}. Therefore, the main task of any projective MOR technique con- 
sists of finding suitable reduction bases V, W € R™™" that span appropriate subspaces 
V = span(V) and W = span(W). 
MOR in mechanical engineering typically aims at achieving the following goals: 
1. Goodapproximation. The reduction technique should yield a ROM which captures 
the most dominant dynamics and well approximates the state vector or input- 
output behavior of the FOM either in the time or in the frequency domain. The 
approximation quality can, for instance, be measured pointwise in time by |ly(t) - 
y-Oll.) or ||x(t) - V xlo or pointwise in frequency by ||G(iw) - G,(iw)I|. using 
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suitable matrix and vector norms (-) = {1, 2, oo, Fro, ...}. Another possibility is to 
use normwise error measures as ||y—Y,||(,) Or ||X—V xilo with (*) = {£y, Lo, Loo» -+ -} 
in the time domain, or ||G — G, ||) with (+) = {H2 Hoo» Hankel ...} in the frequency 
domain. 

2. Preservation of system properties. Basic features of the original model (e. g., sta- 
bility, passivity, second-order structure, port-Hamiltonian structure, etc.) should 
be preserved in the ROM. This requirement is achieved by applying special or 
adapted reduction methods tailored to address these demands. 

3. Numerical efficiency. Model reduction pays off if the benefit of having multiple, 
cheap online evaluations (required, e. g., for design analysis, optimization, and 
control) outweighs the upfront offline cost needed for the computation of the re- 
duced model. Thus, the reduction methods should be as numerically efficient and 
stable as possible. Expensive offline, and especially online, computations should 
be avoided. In addition, reduction approaches should preferably be applicable to 
large-scale models and industrial problems. 


Depending on the application and the characteristic behavior of the FOM that should 
be approximated during the reduction, two categories can be distinguished to meet 
requirement 1: 

(i) Initial condition-state-based reduction. This category is especially interesting for 
mechanical engineering, where the eigendynamics, i.e., the state dynamics de- 
scribed for different initial conditions, are particularly relevant. Modal reduction 
techniques, such as modal truncation [94], mainly focus on the approximation of 
the homogeneous problem, i. e., the eigendynamics of the underlying model. POD 
also falls under this category, since it is based on snapshots of the simulated state 
trajectory. 

(ii) Input-output-based reduction. This category is especially interesting for control en- 
gineering or for problems where inputs and outputs are defined. In this regard, ap- 
proaches such as balanced truncation [78, 44] and Krylov subspace methods [42, 
10] exploit the information contained in the input and output matrices B, C to ob- 
tain a reduced model which is tailored to approximate the input-output behavior. 


To meet requirement 2, so-called structure-preserving model reduction is applied. For 
instance, if the principle of virtual work known from mechanical systems should be 
fulfilled, then the reduction should be performed by a Galerkin projection with W = V 
rather than by a two-sided (oblique) Petrov-Galerkin projection. Note, however, that 
most second-order balancing approaches do not underlie a Galerkin projection. In 
any case, with W = V, the reduced matrices preserve the symmetry and definite- 
ness properties of the original matrices. Furthermore, this choice leads to the foremost 
aim of preserving crucial properties such as the stability, passivity, and structure of 
the original FOM. Note that second-order models could also be reduced by first re- 
formulating them into the state-space/first-order representation (with N = 2N), and 
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then by reducing the first-order models. However, unless special care is taken dur- 
ing the reduction of the reformulated state-space model, it can be difficult to gain 
back a structure-preserving, second-order ROM out of the first-order ROM [102, 103, 
85, 21]. Consequently, MOR for mechanical systems is often performed directly on 
the second-order FOM by applying second-order reduction techniques suited to meet 
the mentioned requirements and needs. Hereby, either (i) classic modal-based pro- 
cedures (such as second-order modal truncation) or (ii) adapted input—output-based 
approaches (e. g., second-order Arnoldi [SOAR] [6, 9, 102], SO-IRKA [112], or second- 
order balanced truncation [76, 108, 17, 16]) can be applied. Note, however, that some of 
these reduction techniques still use the first-order representation to compute, e. g., the 
Gramians or the optimal interpolation data, in order to build afterwards second-order 
reduction bases V, W to project the second-order model. 

It is also worth mentioning that in many engineering applications multiphysics 
problems arise, yielding coupled domain models. Similarly, in structural dynamics 
and industrial applications, the mechanical systems often consist of several separa- 
ble components or substructures. In this context, the concept of substructuring and 
component mode synthesis (CMS) [48, 56, 24, 1] plays a key role, allowing to partition a 
large structural model into multiple substructures that are individually reduced, and 
then compatibly coupled along the component interfaces after reduction. Typical CMS 
techniques are Guyan condensation [48], the Craig-Bampton method [24], and their 
derivatives [56, 1]. Craig-Bampton, e. g., represents the combination of (i) a modal- 
based and (ii) an input—output-based procedure, since (i) “component eigenmodes” 
are combined with so-called (static/dynamic) (ii) “constraint modes” obtained for 
a unit displacement applied to the interface degrees of freedom. In addition to the 
classic CMS techniques, system-level interface reduction approaches that reduce the 
number of interface degrees of freedom are also very common [51]. Finally, note that 
input-output-based reduction approaches such as balanced truncation and Krylov 
subspace methods can naturally be applied instead of the modal-based “component 
eigenmodes.” This has been done, e.g., in [33, 84, 51], where (static/dynamic) (i) 
“constraint modes” have been combined with (ii) “input—output-based component 
modes.” 


2.2 Model order reduction of a thermo-mechanical 
machine tool model aimed for position control 


For modern machine tools, the accuracy, stability, and repeatability are the key perfor- 
mance factors. Thermally induced displacement errors at the tool center point (TCP) 
are among the main causes of work piece defects. The heat generated by motors, the 
friction-caused heat, the influence of the environment temperature, etc., are the fac- 
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tors which lead to unwanted thermal expansion of the machine tool [29, 77, 82] and 
require a real-time compensation (position control). In the past few years, the col- 
laborative research center Transregio 96 [43] contributed a great deal to the optimal 
thermo-energetic design of machine tools. Numerical simulation via, e. g., the FEM is 
a common tool to compute the thermal expansion. However, the accurate numerical 
model is too large to be co-simulated with the control circuitry. 

In the following, the successful application of Krylov subspace-based MOR from 
[39, 6, 102] is presented for creating a reduced-order thermo-mechanical machine tool 
model. Our case study is an academic model, displayed in Figure 2.1. 


Motor Y 
x 


t Iio Moving-stage 
ane 


Motor X 
Tool 


Workpiece } e Base 


Figure 2.1: Structure of the machine tool model. 


For demonstrating purpose, we observe a single stage movement in the positive 
x-direction and a single heat generation in the spindle of the tool. The heat gener- 
ation is due to electrical drive only, whereas the friction-caused heat is neglected. 
Furthermore, ideal thermal contacts and temperature-independent material thermal 
parameters (volumetric heat capacity and heat conductivity) are assumed. The me- 
chanical parts of the model are connected by linear springs, instead of using more 
realistic ball bearings, in which the frictional and thermal effects should be consid- 
ered [65]. These assumptions lead to a linear thermo-mechanical model, which can 
be reduced by Krylov subspace-based MOR. Please note that as long as the above 
linearity conditions are fulfilled, this methodology can be applied to more realistic 
models as well. The primary goal of this section is to demonstrate that the position 
control can be implemented based on such an ROM. Further applications of MOR to 
machine tool models can be found in [32, 96, 81, 68]. 


2.2.1 Coupled domain thermo-mechanical models 


In the thermo-mechanical model, both thermal and mechanical domains are taken 
into account during the simulation. The thermal and mechanical domains can be cou- 


2 Model order reduction in mechanical engineering —— 41 


pled via the input vector or via the system matrices (strong coupling). As the thermal 
expansion is a transient process, which requires real-time compensation, the coupling 
via the system matrices is recommended. Such a strongly coupled linearized machine 
tool model has the following form: 


A ea e A 


UN+k : M D K (2.13) 
q 
= C > 
[z 
where Mg: Dg Kq € R are the mass matrix, the structural damping matrix, and the 


Riek 


stiffness matrix, respectively; D7, Ky € are the specific heat matrix and the con- 


ductivity matrix; Drq € RON is the thermo-elastic damping matrix and K qT € | 


is the thermo-elastic stiffness matrix. Matrices Drg and Ky couple the thermal and 
mechanical domains. The state vector contains two parts, i. e., the structural displace- 
ment vector q(t) € RY and the temperature vector T(t) € IR“. In this model a single 
force input u, is applied onto the moving stage in the positive x-direction and a sin- 
gle heat generation input u, is assumed in the spindle of the tool. The output matrix 
C € R*“*+*) defines the observed outputs, displacement of the moving stage at the 
selected node in the positive x-direction and the temperature at the TCP. The machine 
tool is fixed at the bottom ground and the bottom temperature is set to 20 °C. The heat 
exchange between system and environment is modeled by the convection boundary 
condition @> = A(T -T ambient), Where @z is a heat flux and h is the heat transfer coef- 


ficient, which is set to 5 a. The convection is considered over the whole surface of the 


machine tool model and the ambient temperature is set to 20 °C. The Rayleigh damp- 
ing is considered, that is, the damping matrix D, is proportional to the mass matrix 
M q and the stiffness matrix K q as follows: D; = aM qt pK g In this case, based on 
experience, parameters a = 0 and £ = 1.5915 - 10“ are chosen to cause the damping 
ratio of 1% at a frequency of 20 Hz. 

The advantage of model (2.13) is that all physical effects are included. However, as 
will be shown in Section 2.2.2, MOR of this model leads to unnecessarily large ROMs, 
whose time integration within the controller loop might be prohibitive. On the other 
hand, as the time scale of structural dynamics (= 1s) is much smaller than that of the 
heat transfer (~ 10° s), one might think of separating physical domains and integrating 
them at the system level with different time steps, as suggested in [15]. 

Therefore, the so-called quasi-static approximation of structural mechanics is 
used, in which the mass matrix M, and the structural damping matrix D, from equa- 
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tion (2.13) are ignored. The resulting model has the following form: 


lo ollrl*Lo mallee 


Zy+k-Quasi $ D K (2.14) 
q 
y= Cc > 
H 


and accounts for thermal expansion of the mechanical structure and the transient 
heat transfer. Furthermore, the thermo-elastic damping Drq, which describes the 
influence of structural dynamics upon the temperature field, can be neglected as 
well, because the dissipated heat is negligible compared to the motor-generated heat. 
Please note that the reduction of the DAE system (2.14) with Krylov subspace-based 
MOR is numerically well conditioned, since those methods solely impose that the 
matrix K + SoD is regular for some value sọ of the Laplace variable [6]. 

At the system level, (2.14) has to be combined with the purely mechanical model: 
Sy l Mä + Da4 + K44 = Bgus (2:15) 

y = C44 


that is, with its reduced counterpart, to reflect the dynamical behavior of the structure 
within a control loop. 


2.2.2 Application of Krylov subspace-based MOR 


The main issue in reducing coupled domain thermo-mechanical machine tool models 
lies in the fact that both physical domains have very different time constants, as the 
time scale of the structural part is much smaller than that of the heat transfer. In the 
following, the Krylov subspace-based MOR is applied, which matches the moments 
(coefficients of the Taylor series) of the transfer functions of the full and reduced mod- 
els. For the reduction of model (2.13), the SOAR algorithm from [6] and [102] is used. 
For the first-order system (2.14) and for the proportionally damped second-order sys- 
tem (2.15), the first-order block Arnoldi algorithm from [39] is implemented (for the 
application to (2.15), see [28]). 

Two inputs (mechanical force applied at a selected surface of the moving stage 
and heat source in spindle) and two outputs (displacement at selected node of the 
moving stage and the temperature at TCP) are defined, which leads to the following 
2 x 2 transfer function matrix: 


(2.16) 


G(s) = pe a] 


Ga (s) Gns) 
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where G(s) = FS, Gals) = FH Gals) = GH Gals) = GG and Yaispi(S) 
and Yiemp(s) are ie ae cana of the e and A outputs, 
respectively, defined by matrix C in (2.13); Uforce(S) and Uheat(S) are the Laplace trans- 
forms of the force and heat inputs, respectively, defined as u,(t) and u(t) in (2.13). 

The goal is to investigate the convergence behavior of each reduced model for dif- 
ferent model sizes and compare them with respect to their applicability for position 
control. The relative error between the full and the reduced model transfer functions 
at a specific frequency f is defined as 


ef) = ||GG2nf) - G,(i2nf)||,/||GG2nf)|,. (2.17) 


For the harmonic simulations of the full-scale model, we use the FEM simulator ANSYS 
Academic Research, Rel. 18.0, whereas the reduced models are created with model 
reduction inside ANSYS [98]. 

Firstly, the purely mechanical proportionally damped model (2.15) with the sin- 
gle force input (force applied at a selected surface of the moving stage in the positive 
x-direction in Figure 2.1) and a single output (displacement at selected node of the 
moving stage) is investigated. 

Figure 2.2 (left) shows an excellent match between the transfer function G,,(s) 
of the full model with 51,378 degrees of freedom and the reduced-order 100 model. 
The expansion point 0 Hz is chosen for the Arnoldi-based reduction and for matching 
the frequency range between 1Hz and 1000 Hz, which is the frequency range of in- 
terest for the particular application. Note that other frequency ranges of interest can 
be matched by employing different expansion points [42]. From Figure 2.2 (right), it 
is observed that for matching the transfer function with maximal relative error of 1% 
over the whole frequency range of interest, a moderate number of 30 Arnoldi vectors 
— which represent the orthonormal basis of the Krylov subspace K(-K aM e -K7 Bo) 
— is required if no other expansion points are to be employed. 
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Figure 2.2: Transfer function Gy,(s) of purely mechanical full-scale model (2.15) and correspond- 
ing reduced-order 100 model, gained by Arnoldi-based reduction with expansion point 0 Hz (left). 
Relative error between the transfer functions of full-scale model (2.15) and corresponding reduced 
models of different sizes at different frequencies (right). 
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The coupled domain model (2.13) suffers from different time constants of structural 
dynamics and heat transfer, as the ROM has to simultaneously match the frequency 
ranges from 0 Hz to 10 Hz (thermal expansion effects occur here) and from 1Hz to 
1000 Hz (quick changes in dynamics occur here). 

Figure 2.3 (top) shows an excellent match of the transfer functions G,,(s) and G,,(s) 
of the thermo-mechanical model (2.13) exposed solely to a single unit force input. Fig- 
ure 2.3 (bottom) shows an excellent match of the transfer functions G(s) and G,,(s) of 
the thermo-mechanical model (2.13) exposed solely to a single unit heat input; G(s) 
describes the thermal expansion. In this case we used the unit input for computing 
the transfer function, as the model is linear. At the system level, an arbitrary input 
can be used. 
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Figure 2.3: Transfer functions G,,(s) (top-left), G,(s) (top-right) with single force input and G,,(s) 
(bottom-left), G22(s) (bottom-right) with single heat input of strongly coupled full-scale model (2.13) 
and corresponding reduced models of order 400, gained by the SOAR algorithm with expansion 
points 0 Hz, 10 Hz, 100 Hz, and 1000 Hz. 


In order to match the full required frequency range from 0 Hz to 1000 Hz, with maximal 
relative error of 1%, 300, respectively, 250 Arnoldi vectors and four expansion points 
(0 Hz, 10 Hz, 100 Hz, and 1000 Hz) are required (Figure 2.4). This high number of 250 
vectors is the main disadvantage of reducing the strongly coupled model (2.13). Note 
that more modern methods allow for an automatic choice of the expansion points and 


2 Model order reduction in mechanical engineering —— 45 


10° eeeeeeee 
10? i 
—-—- 0.000599 
107% ---- 0.035938 
10! i 
s $ i — 2.154434 
z z 10? —— 1000.0 
5 10° S EY a : 
a i f EY “Fi 
2 10-14 1 g 10% | 
© A to Eee 
2 of A 
107? 10-77: į CEEA 
ii 
1073 DOAN 2s a Sse EEEE eN 
0 100 200 300 400 
Dimension Dimension 


Figure 2.4: Relative error (2.17) between the transfer functions of full and reduced model at different 
frequencies for different sizes of reduced models with only force input (left) and with only heat input 
(right) in strongly coupled thermo-mechanical model (2.13). 


the corresponding number of moments. In this work, however, we adopted the practi- 
cal engineering approach of “trial and error,” since our goal was to apply the classical 
MOR methodology to our coupled physics, realistic industrial application rather than 
to apply or improve adaptive reduction procedures. Future research should include 
more modern methods based on adaptive approaches such as, e. g., IRKA, TSIA, and 
CUREd SPARK from [45, 27, 113, 85, 111]. 

It is expected that the reduction of the coupled model with quasi-static approxi- 
mation of the mechanical domain (2.14) leads to reduced models with lower order than 
the reduction of (2.13). Furthermore, (2.14) can be reduced with the first-order Krylov 
subspace algorithm from [39]. Figure 2.5 shows an excellent match of the transfer 
functions G,,(s) and G,,(s) of the full-scale and reduced-order 100 thermo-mechanical 
model with quasi-static approximation of the mechanical domain (2.14). It is exposed 
to a single heat input and G,,(s) describes the thermal expansion, which should be 
accounted for within the position control. In this case, however, there is no need to 
cover the frequency range until 1000 Hz, as the structural dynamics are not included. 
Rather, the frequency range of interest from 0 Hz to 10 Hz should be approximated. For 
this, three expansion points (chosen by “trial and error”) are still needed: 0 Hz, 0.1 Hz, 
and 10 Hz. 

Figure 2.6 shows that for matching the transfer function G(s) of the fully coupled 
model with quasi-static approximation of the mechanical domain and the single heat 
input, with maximal relative error of 1%, within the frequency range of interest, solely 
50 Arnoldi vectors and three expansion points are needed, which outperforms the re- 
duced models of system (2.13). 

In conclusion, by introducing the quasi-static approximation of the mechanical 
domain, the accuracy of the reduced thermo-mechanical model could be improved 
and the dimension of the reduced model could be decreased. For the complete ap- 
proximation of the frequency range of interest, this model has to be combined with 
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Figure 2.5: Transfer functions G,,(s) (left) and G,>(s) (right) for single heat input of strongly coupled 
full-scale model with quasi-static approximation of mechanical domain (2.14) and corresponding 
reduced-order 100 models, gained by Arnoldi-based reduction with expansion points 0 Hz, 0.1Hz, 
and 10 Hz. 
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Figure 2.6: Relative error (2.17) between the transfer functions of full and reduced model at different 
frequencies for different sizes of reduced model with only heat input in strongly coupled thermo- 
mechanical model with quasi-static approximation of the mechanical domain (2.14). 


the (reduced) purely mechanical model at the system level, as will be shown in the 
next section. 


2.2.3 Position control scenario based on reduced-order model 


For position control of machine tools usually a cascade controller structure is applied 
[40]. Our goal is to parameterize such a controller based on an ROM. Figure 2.7 shows 
the ANSYS Twin Builder simulation setup. 

In Figure 2.8 (left) the impact of the position control to the moving stage, when 
taking into account only structural dynamics of the machine, is displayed and Fig- 
ure 2.8 (right) shows the gain-parameter optimization of the controller, based on the 
reduced-order purely mechanical model (2.15) with dimension 30. Note that timing 
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Figure 2.7: Co-simulation of the cascade controller and the reduced-order model (ROM) of the ma- 
chine tool in ANSYS Twin Builder. 
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Figure 2.8: Output node displacement of the reduced purely mechanical model (2.15) with and with- 
out controller displayed in Figure 2.7 (left) and optimization of the controller gain based on the 
reduced-order numerical model (right). 


for five simulations with different controller gain values amounts to solely 24s on an 
Intel(R) Core(TM) i7 @ 2.50GHz RAM 8 GB. 

In order to take into account thermal expansion, the reduced strongly coupled 
thermo-mechanical model can be applied within the control loop as schematically 
shown in Figure 2.9 (left). 

The main disadvantage of this approach is that, due to the fact that structural 
dynamics and heat transfer have very different time constants and, hence, a large fre- 
quency range has to be matched, the size of the reduced model is relatively large for a 
system-level simulation (order 250 up to 300, as shown in the previous section). A rem- 
edy is to apply the quasi-static approximation of the mechanical domain, as described 
in equation (2.14). Based on the reduced-order quasi-statically approximated coupled 
model, it is suggested to use the structure for the positioning controller shown in Fig- 
ure 2.9 (right). The advantage over the controller from Figure 2.9 (right) is that both 
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Figure 2.9: Position controller based on the reduced strongly coupled thermo-mechanical model 
(left) and on the reduced thermo-mechanical model with quasi-static approximation of the mechani- 
cal domain (right). 


ROMs are of smaller size and can also be integrated in time separately. Also in this 
setup, it is possible to parameterize the controller based on the ROMs. 


2.3 Coupling of reduced elastic bodies in vehicle 
dynamics 


2.3.1 Requirements on the model reduction software and model 
reduction method 


One important aspect in the development of vehicles is to ensure a comfortable ride 
feeling for the driver. In this context, it is necessary to investigate the effect of external 
disturbances, e. g., from potholes, on the driver in transient simulations over a long 
period of time [72]. Therefore, models of a driver’s seat and a car body are used as an 
industrial automotive example in this contribution. The driver’s seat and the car body, 
the coupling nodes, and the input and output nodes are shown in Figure 2.10. 

In industrial applications, the elastic components are usually modeled with the 
FEM, where complex geometries require fine spatial discretizations to obtain meaning- 
ful models. For the driver’s seat this leads to a model with 1.43 -10° degrees of freedom 
and for the car body to a model with 1.99-10° degrees of freedom. The model properties 
are summarized in Table 2.1. The discretization and model generation are usually per- 
formed with commercial FEM software packages. Obviously, one challenge in MOR for 
industrial models is to access the model data, as the system matrices or geometry in- 
formation from proprietary data structures, for further usage in the actual MOR. In this 
application, the MOR toolbox Morembs is used. It contains interfaces to many commer- 
cial FEM software packages and allows a user-friendly import of the model data. Ina 
further step various MOR algorithms for second-order systems can be applied and the 
reduced models can be exported for further use in an industrial simulation chain. The 
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Figure 2.10: Finite element mesh of the driver’s seat and the car body with inputs, outputs, and in- 
terface nodes, kindly provided by Daimler AG. 


Table 2.1: Model properties of the driver’s seat and the car body. 


Number of elements Number of degrees of freedom Number of inputs 
Driver’s seat 23,968 142,941 24 
Car body 511,156 1,993,167 36 


typical MOR work flow with Morembs consisting of import, reduction, and export is 
illustrated in Figure 2.11. An exemplary MATLAB code for import, reduction, and ex- 
port with Morembs is given in Figure 2.12. The toolbox is freely available for academic 
institutions. Further information can be found in [38] and in Chapter 13 of Volume 3 
of Model order reduction. 
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Figure 2.11: Import, reduction, and export work flow with MOR toolbox Morembs. 


Eee system data 


50 — B.Lohmannetal. 


% start the toolbox 
setupMatMorembs ; 
%⁄ path to data from FEM software 
dataDir = ’FE/VehicleStabilizer’ ; 
% set the interaction nodes 
Interactions(1).node = 13; 
Interactions(1).dof = 1:6; 
% import the system matrices from the FEM data 
sysdata = importFEdata(’workdir’ ,dataDir,’id’,’VehicleStabilizer’,... 
? input’, Interactions) ; 
% reduce the system with a Craig-Bampton approach to a reduced order of 10 
redSysdata = MOR(’sysdata’ ,sysdata, ’redmethod’ ,’Craig-Bampton’, ’nred’,10); 
% export the reduced system for further use in multibody software Neweul-M"2 
ElasticBody = RedExport(’sysdata’ ,redSysdata,’savedir’,pwd,... 


?filename’,’redbody’,’target’,’neweulm2’) ; 


Figure 2.12: Exemplary MATLAB code for import, reduction, and export with Morembs. 


Another challenge is that the reduced component models should be reusable in differ- 
ent mounting situations or product variants. This means that a MOR method must be 
able to perform the reduction on component level first and to deliver reduced models 
which can be combined afterwards with other reduced models, e. g., from a database. 

One well-established method meeting these requirements is the Craig-Bampton 
method as suggested in [24]. This method uses a splitting into an internal part and 
a boundary part of the elastic body. Static constraint modes are then combined with 
fixed interface eigenmodes of the internal part of the elastic body. One feasible mod- 
ern method meeting these requirements as well is the so-called CMS-Gram method 
as presented in [52]. The method uses a splitting into an internal part and a bound- 
ary part, too. Then, a modified input matrix is formulated to allow the application of 
input—-output-based MOR methods for the internal dynamics. In this case, the internal 
dynamics are reduced using a two-step approach. The first step uses moment match- 
ing to reduce the large-scale model to a medium-size model. The second reduction to 
a small system size is done using frequency-weighted balanced truncation. Both MOR 
steps are directly applied to the second-order system described by equation (2.4) to 
obtain a reduced second-order system. First, this obviously simplifies the physical in- 
terpretation of the ROM. Second, in industrial applications it is often even necessary 
to preserve the second-order structure, for example if the ROMs shall be used in a com- 
mercial simulation environment. In the following, both approaches will be compared. 


2.3.2 Comparison of the approximation quality of the noncoupled 
and the coupled system 


First, the driver’s seat as a single component is investigated. The Frobenius norm of the 
transfer function ||G(f)||p,. of the original model and of two reduced models of order 
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Neat = 50 are shown in Figure 2.13. One reduced model is obtained using the Craig- 
Bampton approach while the second reduced model is obtained using the CMS-Gram 
method. It can be seen that all transfer functions are in good agreement. The corre- 
sponding Frobenius norm relative error e(f) = IGC) - G,(/)|lpro/l|G@A) lp, of the re- 
duced models is shown in Figure 2.13 for the frequency range of interest from 0 Hz to 
60 Hz. Both approaches show a good approximation quality in the lower frequency 
range. However, the error of the Craig-Bampton reduced model exceeds 10 % around 
60 Hz, which does not fulfill the accuracy requirements of the ride simulation. In con- 
trast, the approximation error of the CMS-Gram reduced model is about one to two 
magnitudes smaller and, therefore, shows a satisfying approximation quality over the 
entire frequency range. 
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Figure 2.13: Frobenius norm of the transfer function of the driver’s seat for the full model, a Craig- 
Bampton reduced model, and a CMS-Gram reduced model with nseat = 50 and corresponding Frobe- 
nius norm relative errors e(f). 


Another aspect is the improvement of the approximation quality for an increasing or- 
der of the reduced model. For the Craig-Bampton model this can be done by adding 
more eigenmodes to the basis. However, identifying additional eigenmodes which are 
important for the transfer behavior is not always a trivial task, especially for industrial 
models. As for the CMS-Gram approach, the approximation can be simply improved 
by adding more approximated eigenvectors of the Gramian controllability matrix to 
the basis. 
Figure 2.14 shows the Frobenius norm maximum error 


Emax = peas, (ler )- GP )ll ero) (2.18) 


between 0 Hz and 60 Hz for an increasing order n,,; of the reduced system. The er- 
ror for the Craig-Bampton method decreases from Emax = 2.46 : 107 at Ngeat = 50 to 
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Figure 2.14: Maximum Frobenius norm error Emax and mean Frobenius norm relative error €mean fora 
Craig—Bampton reduction and a CMS-Gram reduction from 0 Hz to 60 Hz. 


Emax = 2:19 10°? at Ngeat = 150. The convergence of the input-output error is therefore 
rather slow, which is typical for modal-like MOR methods. However, the CMS-Gram 
method shows a very rapid convergence, leading to an error which is several magni- 
tudes smaller, showing its superiority compared to the Craig—Bampton method. A sim- 
ilar behavior can be observed when investigating the mean error 


1 60 Hz 
Enean = 60 Hz | elf) df (2.19) 
0 


from 0 Hz to 60 Hz, as shown in Figure 2.14. It can be seen that the CMS-Gram method 
delivers reduced models with much faster decreasing mean errors in the frequency 
range of interest compared to the Craig-Bampton method. 

Next, the complete system where the driver’s seat is mounted to the car body is 
investigated. The transfer function of the connected, damped system is depicted in 
Figure 2.15. It can be seen that there are more eigenfrequencies from 0 Hz to 60 Hz 
compared to the single model of the driver’s seat, making a good approximation in this 
frequency range more difficult. The reduced coupled system is derived by a kinematic 
coupling of the reduced model of the driver’s seat to the reduced model of the car body. 
The relative error e(f) for the coupled system is shown in Figure 2.16 for a reduced 
order of Neat = 50 for the driver’s seat and a reduced order of Near = 200 for the car 
body. Since both methods are interface-compatible, both methods deliver satisfying 
coupled ROMs, as well. However, the error of the reduced model obtained by the CMS- 
Gram method is about two to five magnitudes smaller compared to that of the Craig- 
Bampton reduced model. So it can be seen again that the CMS-Gram method delivers 
more accurate reduced models. A convergence analysis for increasing orders of the 
reduced models shows that the approximation error of the coupled CMS-Gram models 
also decreases faster. 
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Figure 2.15: Frobenius norm of the transfer function of the coupled system for the full model, 
a Craig—Bampton reduced model, and a CMS-Gram reduced model reduced model with Nseat = 50 
and Near = 200. 
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Figure 2.16: Frobenius norm relative error e(f) of the coupled system for a Craig-Bampton reduced 
model and a CMS-Gram reduced model with seat = 50 and Near = 200. 


2.3.3 Concluding remarks 


Two MOR methods, the modal-based Craig-Bampton method and the CMS-Gram 
method, are compared in this section in the context of an industrial application. Both 
methods deliver interface-compatible ROMs of single components. This allows the 
user to combine different ROMs to conduct, e. g., product variant studies. However, 
it is also shown that the CMS-Gram method yields ROMs with smaller relative errors, 
confirming the superiority of nonmodal-based MOR methods. Using nonmodal-based 
MOR methods as the CMS-Gram method allows the user therefore either to benefit 
from smaller approximation errors or to use reduced models of a smaller order with 
a similar approximation quality. Therefore, it became clear that it is worth consider- 
ing modern input-output-based MOR methods such as frequency-weighted balanced 
truncation for industrial applications. 
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2.4 Error-controlled model order reduction of an 
elastic crank drive 


A small-scaled, four-stroke internal combustion engine with two pistons in V config- 
uration is depicted in Figure 2.17. The system consists of pistons, the crankshaft, and 
piston rods connecting each piston to the crankshaft. This mechanical model of the 
combustion engine is part of a multiphysics system since the gas force acting on the 
pistons results from a chemical reaction. In the model of this section, the gas force 
is approximated by an analytical function. As mentioned by [84], the elastic effects 
which superimpose the overall rigid body movements have a significant influence on 
the behavior of the crank drive. 


rigid connection elements 


piston rod < 


crankshaft » 


K 

a Figure 2.18: Elastic piston rod of the crank 
Figure 2.17: Flexible multibody system of drive. Rigid connection elements (RBE2) are 
a crank drive as major moving parts of a used to create an interface reduction. 
combustion engine. 


Very often the mechanical parts of the crank drive are modeled as a flexible multi- 
body system (FMBS) with the floating frame of reference formulation. FMBS consist 
of flexible and rigid bodies which are coupled by joints and coupling elements. Ad- 
vantages of FMBSs are their inherently modular fashion and the description of the 
flexible motion with respect to the reference frame, which allows a linear description 
of the elasticity. One single flexible body is described with a nonlinear second-order 
ordinary differential equation (ODE), 

p | Pi | 0 Fa A 

. + ; = ar (2.20) 

Maq) M, Jä] lK) +D] Lha å) 


which can be split into two parts: The part belonging to the motion of the floating 
frame, quantities with the subscript f, and the often high-dimensional flexible part, 
quantities with the subscript e, describing the linear elastic motion with respect to 
the reference frame (see, e. g., [33]). The subparts of the equation of motion are ex- 
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plained, e. g., in [33]. The linear elastic part usually stems from a finite element de- 
scription of continua. Since it is very high-dimensional, it needs to be reduced by 
MOR. The procedure is made in a modular fashion. First, only the linear elastic part 
of (2.20), 


M,q(t) + Deg(t) + Keq(t) = Beug(t), (2.21) 


y(t) = C.q(t) (2.22) 


is considered as a second-order, time-invariant multiple-input multiple-output 
(MIMO) system. All reaction, applied, and coupling forces — the latter especially 
need to be taken into account due to nonlinear rigid body motion - acting on the 
elastic body are considered as inputs B,u,(t) and outputs y,(t) = C,q(t) to the 
elastic body. Using the Laplace transformation, the transfer matrix of the system 
G(s) = C,(s’M, + sD, + Ke) ‘B, is obtained. 

For this second-order system, an appropriate subspace is generated by second- 
order structure-preserving reduction techniques, e.g., by a Galerkin ansatz q(t) = 
Vq,(t), where q, € R”, V € RN*" andn < N. The reduced second-order MIMO sys- 
tem consisting of the matrices {Mer Den Ker} = VHMo De Ke} V, Ber = V™Be, and 
Ce; = C.V is never simulated. Instead, the calculated ansatz space V = span(V) is 
used to calculate the reduced nonlinear equations of motion for one body, 


= hey, d,) 


a 0 | 
+ -A i 
Reve + V'D.V4, ki (yp 4) 


M;, (4) | 
q: 


Mer(qr) v'M.V 


| . (2.23) 


It is worth mentioning that by this MOR procedure a perfect hyper-reduction is 
achieved, i. e., the nonlinear terms in (2.23) only depend on the reduced quantities. 
All quantities labeled [-](q,,q,) depend linearly or quadratically on the reduced elas- 
tic quantities. Different from many other nonlinear MOR procedures, no additional 
hyper-reduction scheme (see Section 2.5 of this chapter of this volume and Chapter 5 
of Volume 2 of Model order reduction) like the discrete empirical interpolation method 
from [22] is necessary. 

The quantity parts of (2.23), e. g., Me(@,,4,) or he (Gy, 4,), are called mass invari- 
ants and can be calculated prior to the simulation (see, e. g., [105]). Therefore, no back 
projection into the original high-dimensional space is necessary during the simula- 
tion, which is one benefit of EMBS simulations. 

The described procedure is applied per body, which allows for a modular setup. 
The single reduced or rigid bodies are later coupled using a minimal coordinate ap- 
proach or Lagrange multipliers (see, e. g., [105]). 

As shown in Figure 2.17 for the crank drive example, only the piston rod is con- 
sidered flexible. In other settings (see, e. g., [84]), other parts of the crank drive, e. g., 
the crankshaft are analyzed. As mentioned in [84], elastic effects and the chosen re- 
duction method are important for realistic simulations for a crank drive system. Due 


56 —— B.Lohmannetal. 


to the fact that the input/output behavior is essential, special care needs to be taken 
to approximate the interfaces in a correct way in the modeling and reduction process. 
Nevertheless, we know that every approximation introduces an error. Therefore, we 
are especially concerned to find measures for the evaluation of the error. 

In the next section, a short explanation about the utilized reduction methods is 
given. Afterward, the error in the frequency and time domains is analyzed. Further- 
more, we will also mention some possible error estimators and error bounds. We will 
finish the section with an outlook. 


2.4.1 Used MOR methods 


Since the piston rod undergoes only small deformations, it is modeled as a linear sys- 

tem. Several linear reduction techniques can be used to approximate the elastic coor- 

dinates of the piston rod, e. g.: 

— Krylov method, i.e., matching the moments of the transfer function at defined 
frequencies up to defined orders [42]; 

- Craig—Bampton, i.e., static interface constraint modes in combination with fixed 
interface normal modes; 

—  CMS-Gram, i. e., frequency-weighted balanced truncation for the internal dynam- 
ics together with static correction modes [52]; 

- POD-Gram, i. e., balanced truncation with POD approximated frequency-weighted 
Gramian matrices [36]. 


All these methods have in common that they can be tuned for specific loading sce- 
narios, e. g., by specifying frequencies in the Krylov method. In the previous section 
some information about the Craig-Bampton and CMS-Gram methods is given. Fur- 
thermore, the same work flow is used. For MOR of EMBS a correct consideration 
of boundary conditions and static correctness are essential steps; therefore, CMS- 
based or interpolation-based methods are favored. For systems with many connection 
points, e. g., gearboxes, the Krylov reduction or the CMS-based reduction turns out 
to be challenging. Therefore, an interface reduction is necessary very often in a first 
step. Here a model-based interface reduction is performed by inserting rigid connec- 
tion elements (RBE2) at the two bearings of the piston rod (Figure 2.18). The slave 
nodes (nodes at the bearing seats) and the master node (the central node) behave like 
a rigid body. 

Due to the boundary conditions, the connecting points at the piston rod can only 
move in the yz-plane. Therefore, only the inertia forces in these directions are con- 
sidered as excitations in the CMS-Gram method and collocated outputs C, = B! are 
considered. The six static modes of the interface node between crankshaft and piston 
rod correspond to the rigid body modes. For the CMS-Gram approach, the frequency 
range of interest is Zp = [0 kHz, 8 kHz]. 
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2.4.1.1 Results in the frequency domain 


We are interested in the elastic movements; therefore, we only analyze the nonrigid 
body movement, which is the elastic dynamics of the system. This neglects the er- 
ror due to coupling of the rigid body movement and internal dynamics. In a first step 
the separation between interface and internal nodes is conducted by a CMS-based ap- 
proach, and later the Gramian matrix is used to approximate only the internal dynam- 
ics, which belongs to the internal nodes. In Figure 2.19, the transfer function of the 
unreduced elastic dynamic of the piston rod is plotted. The error system G, = G - G, 
with the reduced second-order system G,(s) = C,,(s*M.,+SDe,+Ke;) Be, is used to 
evaluate different reduction methods. As in Section 2.3, the relative error 


els = iw) = ||G(w) - G,(@)llgro/IG(@) gro (2.24) 


over the angular frequency w measured in the Frobenius norm of models reduced with 
different reduction methods is plotted in Figure 2.20. The interesting frequency range 
between 0 kHz and 20 kHz is equidistantly sampled with 500 points. 
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Figure 2.20: Different relative errors for 
different reduction methods and different 
reduction levels. Blue: CMS-Gram; green: 
classical Craig—Bampton. Lighter colors label 
models with smaller reduction size. 


Figure 2.19: Norm of the transfer function of 
the unreduced (N = 7332) internal dynamic 
(black) and with CMS-Gram (n = 6) reduced 
internal dynamic (blue). 


It can be seen that the relative error for the CMS-Gram-based reductions is smaller 
than the error of the Craig-Bampton reductions. 

If we compare the CMS-based approach with reduction not based on a separation 
between interface and internal nodes, e. g., modal or POD-Gram reduction, a far worse 
approximation of the elastic/internal dynamics is achieved even with higher dimen- 
sions. The eigenvectors of the free system have a very low correlation to the internal 
dynamics. The generalized inertia forces do not excite these eigenmodes. Therefore, 
they are unimportant for the problem at hand. The POD-based reduction is not tuned 
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to approximate the internal dynamics. Therefore, no fair comparison is possible and 
we decided not to show these results. 

Typically, the error is measured in specific norms, e. g., the H,,- or the H,-norm 
[85]. For Gramian-based reductions, an error bound based on the sum of neglected 
Hankel singular values (HSVs) is available. 

In Figure 2.21, the relative error in the #,-norm of the internal dynamics is plot- 
ted. For a reduction size of 6, only static modes are used; therefore, the CMS-Gram 
and the Craig-Bampton approach behave exactly the same. Already with one more 
reduction mode, the CMS-Gram approach shows better results. After 12 and 16 modes, 
we again see a big improvement (Figure 2.21). The Craig-Bampton reduction has a 
step-like decay, some eigenmodes of the bounded system have only a minor influence 
on the approximation quality. Therefore, after we achieved a reduction size of 14 no 
further improvements were made with the Craig-Bampton approach. In comparison 
to the Craig-Bampton reduction, the CMS-Gram-based reduction explicitly considers 
the generalized inertia forces, and therefore has a more steady and rapid decay of the 
error and the HSVs. A more elaborate description of this behavior is given by [51] fora 
slightly different crank drive example. 
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Figure 2.21: #/,-error of the internal dynamics. 


We saw that Gramian-based approximation in combination with static condensations 
is favorable. However, for a large-scale system, a direct calculation of the Gramian ma- 
trices is not possible. Two of the many approaches to calculate Gramian matrix-based 
reduction spaces are (i) the POD-based approximation of the frequency-weighted 
Gramian matrices and (ii) a two-step approach in which a Krylov-based approach 
calculates an intermediate size model in a first step. The reduction quality of the POD 
approach depends on the location of the frequency snapshots and the Krylov-based 
reduction depends on the location of the expansion points. The intermediate models 
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need to guarantee an acceptable approximation quality; however, due to their large 
size we cannot calculate the H,,-norm, the H -norm, or €,. 


2.4.1.2 Error estimators in the frequency domain 


As mentioned in [33] and [36], error estimators can be used to gradually select good po- 
sitions of the frequency snapshots — respectively expansion points — at the most suit- 
able position and estimate the error. Different error estimators for a given frequency 
range [Wins Wmax] are available to replace the time needed to evaluate the original 
system in calculating the relative error e, defined in (2.24). 

One possibility introduced by [42] is the replacement with a second reduced sys- 
tem G. The approximation error is then estimated by the relative error between the two 
reduced-order systems (see, e. g., [42]): 


€Grimme(W) = ||@(W) - GW) | p,5/|E@)| ro: (2.25) 


A second error estimator was introduced by [13]: Instead of using a second ROM, 
the ROM from the previous iteration G,_, is used: 


egr) = Gif) - G,_,27f)||-,5/||G<G27/) le, (2.26) 


In the provable error estimator of [62], the error is separated into two complemen- 
tary subspaces Sg and S, of the eigenmodes in the interesting frequency range. This 
error estimator has a provable error bound for lossless systems in addition to an ex- 
tension to second-order systems (see, e. g., [33]): 


K.B. — K,,(w)VL, (iw)Be,. (2.27) 


€xrp(@) = V2(R. N, Spirax) 


The error can be calculated efficiently by an offline-online decomposition. The term 
K., is independent of w and V, the term K,,(w) is independent of the reduction ma- 
trix V, the term I is based on reduced quantities and is further explained in [33], 
and the term V,(R,7,é,__.) is a maximum bound of the magnification function of a 
second-order elementary vibrating system well known in linear vibrations theory (see, 
e. g., [79]). 

In Figure 2.22, the different error estimators are plotted. These error estimators 
could, e. g., be used in a greedy-based selection process to add new expansion points 
for a Krylov subspace used for reduction or in a POD-like approximation of the second- 
order frequency-weighted Gramian matrices [36]. For this example, the error estimator 
€xpp has a slightly different behavior from the other error estimators but both share the 
same form. 

Mechanical systems are not very sensitive to the location of the expansion points 
as other systems may be, especially in this example, where there is not much dynamic 
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Figure 2.22: Comparison of the three error estimators and the relative error. 


in the interesting frequency range. Therefore, all of the error estimators are helpful in 
automating the reduction process. Even if the absolute values of the error estimators 
are not consistent with the real error, the frequency location of maximum values and 
the convergence behavior of the error estimators are consistent with the real error. 


2.4.1.3 Results in the time domain 


In the time domain, we compare the well-established Craig-Bampton method with 
a mediocre basis size of 25 to the other reduction methods with basis size of only 7 
(Figure 2.23). The Craig-Bampton model of size 25 is considered as a converged ROM. 
For the results an explicit first-order Runge-Kutta solver with a fifth-order automatic 
time step size control for stiff ODEs (MATLAB ode15s) is used. Modal reduction shows 
a phase shift and a larger error at the time of highest deformation. Despite having 
no phase shift, the Craig-Bampton method with only seven basis vectors shows bad 
conformance with the reference solution at the zoom-in view due to the small basis 
size. Only CMS-Gram accomplishes almost no error compared to the reference solution 
with a basis size of only 7. 


2.4.1.4 Error estimators in the time domain 
Error bounds based on the residual 


R,,(t) = M.Vq, + D.Vq,(t) + K.Vq,(t) - Beue(t) (2.28) 


between the reduced and the original model can be used to deliver a posteriori error 
bounds in the time domain, which account for the current excitation. The general, 
a posteriori error estimator A of [99] for second-order mechanical systems is used in 
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Figure 2.23: Deformation of one node in the z-direction for various reduction methods. The zoom-in 
view shows the time of the largest deformation. 


the following. It has been analyzed in [37]. We apply this error estimator to the single, 
clamped piston rod excited with an approximated gas force. The true error is compared 
to the error estimator in Figure 2.24 for different model reduction methods. 

It is of no surprise that modal reduction with 100 modes produces the smallest 
error. Looking at reductions with smaller basis size, Krylov 6 seems to have the best 
results. While all other methods have a practically indistinguishable error bound, the 
error estimator of Krylov 6 is two orders lower. This phenomenon still needs to be 
investigated. 


2.4.2 Outlook 


For EMBS simulations, MOR is one essential step to create fast-to-calculate but still 
convincing simulation models. If there are many inputs to the system, the interface 
reduction process often plays a far more critical role than the used MOR schemes. Error 
estimators in the frequency domain are helpful in automating the reduction process. 
Frror estimation in the time domain with a priori error bounds for coupled multibody 
systems is a nontrivial task since coupling terms influence the behavior of a single 
part. New strategies such as to rewrite the multibody system as DAEs to consider all 
reaction forces as inputs need to be developed and rigorously tested. 

With the CMS-Gram methods, a method is at hand which combines the benefit of 
static correctness, with an error-based, Gramian matrix-based approximation of the 
internal dynamics. Furthermore, the generalized inertia forces are considered in the 
reduction process. Good results are achieved in the frequency as well as in the time 
domain. 
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Figure 2.24: True error A(t) = ||Vq,(t) — q(t)ll2 in solid lines and the error estimator Ay of the state 
in dashed lines of a clamped piston rod excited with an approximated gas force; A u represents the 
error estimator of all methods except Krylov 6. 


2.5 Nonlinear model order reduction for a leaf spring 
model 


2.5.1 Nonlinear model reduction in structural dynamics 


In engineering, one finds many applications where exact knowledge of the dynamic 
behavior of the structures during operation is essential. This is mainly due to the fact 
that the dynamics influence design goals such as fatigue, vibration comfort, or noise 
emission. In this section, we present the MOR of the leaf spring of a truck. This part 
undergoes large deformations in certain maneuvers like strong braking and exhibits 
hence geometrically nonlinear behavior. Since the transfer path of the excitation force 
goes through the spring, an accurate model of the nonlinear spring is crucial for the 
dynamics assessment of the otherwise linear chassis model. 

The semi-discretized equations of motion from finite element models that are able 
to describe the dynamics of structures with large deflections are described by equa- 
tion (2.2). To reduce the computational effort, one needs to perform two steps. First, 
a Galerkin projection is carried out in order to reduce the number of unknowns. As- 
suming a linear viscous damping matrix, this yields 


V'MV4q,(t) + V'DV4q,(t) + Vf (Vq,(t)) = V'BF(t). (2.29) 


This approximation is similar to the projection methods used in linear model re- 
duction for structural dynamics as described in Section 2.1. However, this reduction 
step alone does not reduce computation time significantly. The solution time for the 
leaf spring model, which will be shown below, can only be sped up 1.6 times by this 
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step, despite the fact that the solution vector has been reduced from 200,000 degrees 
of freedom to 100. The reason for this is the nonlinear restoring force term V'f (Vq), 
which is evaluated by an assembly of all element forces. Unlike in the reduction of lin- 
ear systems, one cannot build a reduced matrix since the restoring forces are nonlin- 
ear. Hence, the evaluation of the nonlinear term is the new bottleneck of the solution 
process. The methods reducing the evaluation costs of this nonlinear term are called 
“hyper-reduction” methods. The application of a hyper-reduction is the second step 
that needs to be performed. Different hyper-reduction methods can be found in the lit- 
erature. One prominent example is the discrete empirical interpolation method [22]. 
Other approaches are more suitable for nonlinear structural dynamics, such as the 
energy-conserving sampling and weighting (ECSW) method [31, 30] and polynomial 
expansion [50], which will be used here. 


2.5.2 Basis generation 


One approach to get a reduction basis is the so-called POD. It is based on the singular 
value decomposition q, = U EWT of some training displacements q,, which usually 
come from results of a time integration of the full model. The reduction basis V is then 
built by stacking the first n left singular vectors into V so that V consists of the first n 
columns of U. 

Another approach generates the reduction matrix from two parts, 


V= [Viin V al (2.30) 


where Vin contains some vibration modes @ of the linearized system and V contains 
some modal derivatives v,;. The modal derivatives describe how the mode @; changes 
if the system is perturbed in the direction of another mode @,. A slightly modified 
version of the modal derivatives are the static modal derivatives (SMD) [107, 57] that 
are calculated by solving 


Kv; S -Ve KDP; (2.31) 


which often perform better than the modal derivatives. 


2.5.3 Hyper-reduction 


2.5.3.1 Polynomial expansion 


When linear materials are used, the internal force taking into account geometric non- 
linear effects due to large deformations and rotations can be written using Einstein 
summation convention as 


fi(@) = KPa; + K} djar + KG ajakan (2.32) 
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Thus, the nonlinear restoring force vector can be described by three tensors K®?, K®, 
and K®), 
Applying a Galerkin projection with the reduction basis V, one gets 


5, (1) z> (2) z (3) 
(V'F(Vq,)); = Kij drj + Kik Wj Iek + Kiia Vj Uk Unb (2.33) 


with 


Kix 7 (V"), Kip Ving Vink? 


lmn 
Roa = (V") in Koop V xj Vok Vor 

Therefore, one only needs to identify K S k° ,andK © to get a reduced model whose 
nonlinear term can be evaluated very fast. For the computation of these three tensors, 
several techniques exist which can be classified as intrusive and nonintrusive meth- 
ods. Intrusive methods identify the tensors by computing the coefficients on element 
level [92, 106] requiring access to the element formulation inside the finite element 
code. Nonintrusive methods, however, do not require the element formulation and 
can hence be used with commercial finite element software, where the access to inter- 
nal computations is limited. Some techniques identify the tensors by prescribing dis- 
placements and evaluating the resulting nonlinear forces [80, 61], others by prescrib- 
ing forces and evaluating displacements [74]. The implicit condensation and expan- 
sion method [50] computes both polynomial tensors and the reduced basis in one step. 
In our case study, we use the nonintrusive identification as proposed in [101, 86, 87], 
which uses multiple evaluations of the reduced tangential stiffness matrix K(q) at dif- 
ferent displacements. 


2.5.3.2 Energy-conserving sampling and weighting 


Another approach is the ECSW method [31, 30], which is based on a reduced assembly 
of a subset of the elements and extrapolates their contribution to the full force vector: 


V"F(Va,) = Y VL (LeVeg:) = X E V'LfelLeV eq). (2.34) 
ecE ecECE 

The matrix L, is a Boolean mapping matrix from the local degrees of freedom of ele- 
ment e to the global degrees of freedom and , are positive weights for extrapolation. 
The weights &, and the element subset E are chosen by using training displacements 
for which the virtual work of the restoring force in the direction of the reduction basis 
shall be retained in the hyper-reduced model. As in the POD, these training displace- 
ments are often computed from simulations of the high-dimensional model. Another 
approach to gain training displacements is presented in the next section, since per- 
forming full simulations is numerically expensive. 
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2.5.3.3 Nonlinear stochastic Krylov training sets for ECSW 


An alternative approach to build training vectors for ECSW, which avoids the full simu- 
lations, is called nonlinear stochastic Krylov training sets (NSKTS), proposed in [100]. 
The idea of this method is to build a subspace whose vectors are able to approximate 
the nonlinear force vector f . If the viscous damping term D is neglected, one finds 


Mä(t) + f(q(t)) = BF) ~ f € span(B, MG(t,),MG(t,),....Mq(t,)). 


As the accelerations q(t;) are unknown, an approximation for the subspace described 
above must be made. In the NSKTS, the subspace is approximated with the Krylov 
subspace 


Fy = span{B, MK~'B, (MK™')’B,...} = K(MK™,B), (2.35) 


which is orthogonalized and normalized so that F ay -IF kry = L. 
Then, some random vectors f hsgrs living in this Krylov subspace are generated 
and applied as external force to the nonlinear static problem 


f(a”) =kfNsxrs» K € (0,1). 


This equation must be solved by a nonlinear solver such as Newton—Raphson. The 
training set is then built by saving the solution q® for some load steps k for each ex- 
ternal force fġsgrs as training vector. This procedure reduces the computation cost for 
obtaining a set of training vectors by avoiding direct time integration of the full model. 
All steps can also be carried out in the reduced subspace spanned by V. This has the 
advantage that the nonlinear static problems can be solved faster and the resulting 
training vectors live in the subspace of the displacements for the ROM [100]. There are 
no a priori error estimates for the NSKTS method and, therefore, the necessary number 
of training vectors is evaluated by varying its number and checking the convergence 
of the solution. 


2.5.4 Case study: leaf spring 


In the following, the results of the reduction of a truck chassis leaf spring are sum- 
marized. The case study is carried out with the Finite Element Package AMfe, devel- 
oped by the Chair of Applied Mechanics at Technical University of Munich. The code is 
available at https://github.com/AppliedMechanics/AMfe. The full study is published 
in [101]. 

Figure 2.25 shows the leaf spring, which consists of two leafs (top and bottom), 
a central fixture that joins the leafs, and two rubber pads that keep the distance be- 
tween the leafs at the ends. The front eye is fixed with the frame via a joint allowing 
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Figure 2.25: Leaf spring obtained from [101] consisting of two leafs, rubber pads, and central fixture. 


rotations about the y-axis while the rear eye allows both rotations about the y-axis and 
a translation in the x-direction. The rubber pads are fixed with the top leaf and have a 
sliding contact with the lower leaf. The model is meshed with 85,762 linear elements 
(tetrahedrons and hexahedrons). A load case is applied that stems from a multibody 
simulation of a brake maneuver. Time-varying forces and loads are applied on the top 
of the central fixture. 

We compare the simulation time and accuracies of a simulation with 1,500 time 
steps carried out with a generalized a scheme (pœ = 0.8). As accuracy measure we 
define the relative error 


di Aq(t;)'Aq(t;) 


= -100% with Aq(t,) = q(t.) —rer(t). (236) 
VZ ret(ti)' Aree (ti) 


We compare two kinds of errors: First, the error of the Galerkin projection, where the 
non-hyper-reduced but projected system is compared with the full solution (RE;), and, 
second, the error of the hyper-reduction, where the hyper-reduced solution is com- 
pared to the non-hyper-reduced but projected solution (RE),,). 

Table 2.2 lists the relative errors and simulation times for different simulations. 
The wall time for a full simulation run with 216,499 degrees of freedom is 40,022s 
(= 11h). As reduction basis, 100 basis vectors consisting of 40 vibration modes and 
60 static modal derivatives are chosen. One can see that only a small speedup of 1.66 
is gained while the error is quite small. To further speed up the simulation, hyper- 
reduction is needed. 

Therefore, two hyper-reduction methods are considered. First, the ECSW is car- 
ried out with 160 nonlinear stochastic Krylov training sets. The hyper-reduced mesh 
is shown in Figure 2.26. Only 816 elements are evaluated while the relative error of the 
hyper-reduction is 0.13%. The hyper-reduced model gains a speedup of 38.23 com- 
pared to the full simulation. Second, the polynomial expansion is carried out. The 
identification of the tensor coefficients needs 28,856 s (= 8h), but the hyper-reduced 
model has a wall time of just 113 s, which is a speedup of 354.18 compared to the full 
simulation. Since the model has a linear elastic material and the nonlinear strain mea- 
sure is quadratic in q, the cubic polynomial expansion of the internal restoring force 
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Table 2.2: Errors and speedup rates of the different reduction techniques for the leaf spring example 
obtained from [101]. 


Reduction method DOFs Elements RE,[%] REp,[%] ty[s] Speedup [-] 
full 216,499 85,762 = - 40,022 = 
Modes & SMDs 100 85,762 0.68 = 24,127 1.66 
Modes & SMDs + ECSW with 100 816 0.78 0.13 1047 38.23 
NSKTS 

Modes & SMDs + polynomial Exp. 100 = 0.68 0.00 113 354.18 
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Figure 2.26: Hyper-reduced mesh obtained from [101]. The reduced mesh consists of 816 elements 
that have different weights ¢. 


(2.32) is exact. Hence, the relative error of the polynomial expansion hyper-reduction 
method is zero in this case. 


2.5.5 Conclusion 


The geometric nonlinear leaf spring has been reduced by a reduction basis of dimen- 
sion 100. The reduction basis has been computed by using the properties of the sys- 
tem without the need for results from full dynamic simulations. The combination of 
40 vibration modes and 60 static derivatives leads to accurate results although the 
full model has 216,499 degrees of freedom. Then, two different hyper-reduction tech- 
niques, ECSW and polynomial expansion, are carried out. Nonlinear stochastic Krylov 
training sets are used for ECSW which avoid full simulation runs and lead to rela- 
tive small reduction costs of about 73 minutes. The ECSW-reduced model leads to a 
speedup factor of 38.23. The polynomial expansion gives the best speedup, which is 
about 10 times higher than with ECSW, while the reduction time is much higher (about 
8h). Thus, one can conclude that polynomial expansion is best suited if offline costs 
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do not matter or if one needs to run many simulations or many time steps with the 
reduced model. However, the polynomial (cubic) expansion is only valid for models 
with linear elastic materials. Another issue is the computational cost and memory de- 
mand for the identification of the polynomial coefficients. Both depend highly on the 
dimension n of the reduction basis, since they increase with O(n"). In our test case, the 
reduction basis of dimension 100 was suitable. For models requiring a larger reduced 
basis, hyper-reduction using the ECSW is a good option. 
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Elke Deckers, Wim Desmet, Karl Meerbergen, and Frank Naets 
3 Case studies of model order reduction for 
acoustics and vibrations 


Abstract: This chapter presents several case studies to illustrate specific aspects in 
setting up reduced-order models of acoustic and vibration models in mechanical ap- 
plications. Modal truncation approaches have been a proven workhorse for over half a 
century in civil and mechanical engineering, but, for many (recent) applications, these 
techniques are too limited. In mechanical engineering, model users are interested ina 
range of model applications: frequency and time domain, linear and nonlinear, single 
domain and multiphysics, etc. This broad range of applications makes it particularly 
challenging to devise appropriate reduced-order model schemes, as a scheme for one 
model use might be completely inadequate for other applications. Krylov methods for 
example have been a go-to technique in many domains, but face particular challenges 
in mechanical finite element models as the system’s eigenvalues lie along the imagi- 
nary axis and the high frequencies are irrelevant for a given mesh size from a physical 
perspective. In the current chapter we explore these particularities for different types 
of mechanical models and simulation purposes, in order to surface several good prac- 
tices and points of attention when applying model order reduction on these models. 
We bring together two different viewpoints: the application of model order reduction 
from a purely mathematical point of view and the physical interpretation of models 
and expected properties of reduced-order models based on physical arguments from 
the field of mechanics. While we touch upon a range of novel model order reduction 
techniques, we do not discuss parametric model order reduction as it is expected that 
the presented guidelines can be exploited in parametric problems without additional 
specific concerns. 


Keywords: Model order reduction, acoustics and vibrations, finite element method, 
structure-preserving methods, nonlinear frequency dependency 
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3.1 Overview of mechanical vibration and acoustic 
applications and models 


3.1.1 Introduction 


The modeling of the vibrational and acoustic behavior of physical systems is far from 
trivial. In a general coupled vibro-acoustic system, in which a structure and acoustic 
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cavities mutually interact with each other, the system behavior is typically determined 
by the coupled response of each of the components, often requiring an accurate rep- 
resentation of the interface. For acoustic simulations, the frequency range of interest 
typically runs up to 20 kHz. In real life, however, this range cannot be covered due 
to the limitations of the current computer-aided engineering tools. Moreover, there 
is a significantly different response in different frequency regions. In general, three 
different frequency regions can be identified, also visualized in Figure 3.1, which are 
typically problem-dependent: 

Low-frequency range In the low frequency-range, the characteristic length of the 
problem under study is smaller than or in the same order of magnitude as the 
dominant physical wavelengths in the dynamic response. In this frequency range, 
the response of the system is determined by well-separated eigenvalues or modes 
and can be predicted by means of deterministic approaches. For vibro-acoustic 
problems, element-based techniques, such as the finite element method (FEM) 
[62] and the boundary element method [57] are most commonly applied. Element- 
based approaches divide the problem domain or its boundary into a large number 
of small elements. Inside these elements, the field variables are approximated us- 
ing simple, often polynomial, functions. As wavelengths shorten with increasing 
frequency, the element size also needs to decrease to diminish the effect of inter- 
polation and pollution errors [21, 34]. As a consequence, the number of degrees of 
freedom increases, as does the size of the system matrices, limiting the practical 
use of element-based approaches to lower-frequency applications. 

High-frequency range When the characteristic length of the problem under study is 
much larger than the dominant physical wavelengths in the dynamic response, 
the considered problem is situated in the high-frequency range. Typically, the 
modal density and modal overlap are high and the system is very sensitive to small 
variations in for instance material properties and geometrical details. As small 
variabilities are inevitable in real-life applications, the response of one nominal 
system loses its meaning. As a result, the spatially averaged response of a number 
of realizations is of interest together with its variance. In this frequency range, sta- 
tistical techniques are applied; for instance the statistical energy analysis (SEA) 
[32] is often used for vibro-acoustic analysis. The SEA divides the problem domain 
into a small number of subsystems in which a spatially averaged estimate of the 
energy level is obtained. SEA is computationally not demanding, but relies on a 
number of assumptions, such as for instance a high modal overlap and an ener- 
getic similarity of the different subsystems. Since these assumptions are only met 
above a certain frequency limit, the method is restricted to the high-frequency 
range. 

Mid-frequency range In-between the low- and the high-frequency range, a fre- 
quency band exists for which it is stated that currently no mature and adequate 
prediction techniques are available. However, for many applications, this mid- 
frequency gap coincides with the frequency range where the human perception 


3 Acoustics and vibrations —— 77 


IF| 


Frequency 


Low frequency range Mid frequency range © ` High frequency range 


Figure 3.1: Typical frequency response function of weakly damped mechanical vibrational system 
[56]. 


and hearing is highly sensitive. Therefore, solutions are sought to bridge (part 
of) this gap. One approach is to increase the frequency range of the deterministic 
approaches by increasing the size of the resulting models. Here, amongst others, 
model order reduction can be an important enabler. 


This chapter will focus on model order reduction techniques for vibrational and acous- 
tic systems in the low- to mid-frequency range. Typically, these finite element models 
of vibrational and acoustic systems result in a system of second-order ordinary dif- 
ferential equations with large, but sparsely populated system matrices that allow for 
efficient solution algorithms. If no complex damping treatments are considered, the 
obtained models are linear and frequency-independent. Finite element models can 
easily be represented as linear state-space systems. The system matrices of a bound- 
ary element model are in general smaller (as only the boundary of the domain has 
to be discretized into elements resulting in a substantially lower amount of degrees 
of freedom) than their finite element counterpart. However, the boundary element 
matrices are fully populated and have a rather complex frequency dependence. For 
this reason, it is not straightforward to convert boundary element equations to a time- 
domain equivalent. This sometimes makes this approach inadequate for engineering 
applications. 

Given the properties listed above, the maturity and the widespread industrial use 
of the FEM make it very accessible for practical problems. This is why only finite ele- 
ment models are considered in this chapter. 

Besides component model analysis acceleration, reduced-order vibrational and 
acoustic models are a key enabler for many integrated simulation applications. In 
flexible multibody simulation, reduced-order vibrational component models are ex- 
ploited in order to allow for the inclusion of coupled body flexibility in a mechanical 
system level context, as discussed in Chapter 2 of this volume. The analysis and design 
of new materials requires multiscale simulation where model reduction has the poten- 
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tial to allow to bridge these different scales in a fully coupled framework [61]. Overall, 
these reduced-order model (ROM) approaches can be considered as a key enabler for 
current and future applications where vibro-acoustic models need to be evaluated in 
a broader system-level performance context. 


3.1.2 Mathematical models of vibrational, acoustic, and 
vibro-acoustic systems 


3.1.2.1 Vibrational problem definition 


By combining constitutive equations based on Hooke’s law, which expresses that the 
relationship between stress and strain is linear, and momentum equations, the elastic 
wave equation is obtained [5]: 


2> 
poe = Ay + MAG) - WG xT xii E, Ga) 
in which ù describes the solid displacement in the three spatial dimensions, F, p is the 
body force per unit volume acting on the solid, and A, and u; are the Lamé coefficients 
of the isotropic material. 

For some commonly encountered geometries in vibrational problems, this elastic 
wave equation can be simplified. If the geometrical domain is much smaller in one 
direction than the other two, Kirchhoff—Love plate theory [51] can be applied, consid- 
ering only bending due to transverse loads for plates subject to small deformations. It 
is explicitly assumed that straight cross-sections remain straight under deformation, 
including no shear effects. In this case, the equation of motion is given by 


dtu atu ðu du 
2 Z Fi 3.2 
& t aea * F) Pap t Ga 


where uis now only the (scalar) transverse displacement, x and y describe the in-plane 
location, F, is the transverse load expressed as a force per unit area, p is the mass per 
unit area, and D is the plate bending stiffness, defined as 


Ek? 


D = 12(1 - v2) > (3.3) 


where h is the plate thickness, E is the plate material Young’s modulus, and v is its 
Poisson’s ratio. As the Kirchhoff-Love plate equation (3.2) is a fourth-order differential 
equation, two boundary conditions have to be applied at each location on the bound- 
ary to have a well-posed problem. Commonly applied boundary conditions are free 
edges, clamped edges, and simply supported edges. Next to plates, often shells are 
applied. The difference with plates is that for shells also in-plane deformations and 
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stresses are modeled. These types of models are often used to represent thin-walled 
structures, increasing computational efficiency. Thin-walled structures are often en- 
countered in vibro-acoustic applications. Shell structures are in general quite stiff in- 
plane, and more flexible out-of-plane. These types of structural models are typical for 
mechanical and civil engineering applications and are not often encountered in other 
disciplines. They moreover tend to couple strongly to the acoustic domain. 


3.1.2.2 Acoustic problem definition 


The constitutive equation in acoustics is the so-called acoustic wave equation: 


2 

Srp- a5 = pot, (3.4) 
which has to be solved in order to obtain the sound pressure p(x, y,z,t) in a given 
system, being the pressure perturbation around the ambient reference state po. In this 
equation, = is the Laplace operator which is defined as =a = = + x + = Po is the 
ambient fluid density, q is the flow rate (m?/s) of an acoustic volume source, and c is 
the speed of sound, defined as 


C= — 3 (3.5) 


where y is an inherent property of the studied gas, being the ratio of the specific heat 
capacity for constant pressure and the specific heat capacity for constant volume. 

The acoustic wave equation (3.4) assumes that the fluid behaves as an ideal gas, 
that pressure changes in acoustics are adiabatic, and that the fluid flow is inviscid. 
More details can be found in, e. g., [13]. 

Acoustics are often studied in the frequency domain. Assuming a time-harmonic 
e/'-dependence of the dynamic quantities and excitations, the inhomogeneous 
Helmholtz equation is retrieved [43]: 


F pw) + p(w) = -spowqlw), (3.6) 
where 
= — (3.7) 
is the acoustic wavenumber at frequency f. The acoustic wavelength A is defined in 


terms of the speed of sound and the frequency as 


Cc 


A= . 
7 (3.8) 
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The Helmholtz equation (3.6) is a second-order differential equation, meaning that 
one boundary condition has to be specified at each point of the boundary in order to 
obtain a well-posed problem. Typically, for bounded acoustic problems, Neumann, 
Dirichlet, and Robin boundary conditions are applied. 


3.1.2.3 Vibro-acoustic coupling 


When strong vibro-acoustic coupling is considered, it is assumed that vibrations in- 

duce acoustic waves, but that the acoustic field in turn also excites structural vibra- 

tions. This is often the case for closed cavities and thin-walled structures, as often 

encountered in mechanical applications like car interior cavities. In a coupled vibro- 

acoustic system, the pressure field p in the acoustic domain and the elastic displace- 

ments ù} mutually affect each other on the wetted interface: 

— the structural out-of-plane velocity ù on the wetted structure is seen as an imposed 
velocity for the acoustic domain; 

— the acoustic pressure field p on the wetted structure is considered as a distributed 
load on the structural domain. 


3.1.3 Finite element modeling and discretization 


3.1.3.1 General formulation 


The modeling procedure of the FEM can be applied to any general set of (coupled) 
differential equations. In a first step, the problem domain is discretized into a large 
number of small elements which are interconnected by a network of n;, nodes. Each 
field variable v;(7) at location 7 is approximated in each of the elements by a solution 
expansion Ŷ;(ř) in terms of nge (polynomial) shape functions Np: 


vi) = HF) = È NOg 
fil 
= N,(7)vj. (3.9) 


The nodal values v; belonging to each of the nfe nodes are gathered in the vector of 
the (generalized) degrees of freedom v;. The row vector Ñ, collects the ng shape func- 
tions N;. These shape functions only have a nonzero value inside the element to which 
they belong. Moreover, each shape function has a value of 1 for only one degree of free- 
dom of the element and is zero at all others. The polynomial shape functions do not 
exactly satisfy the differential equations describing the physical problem to solve, nor 
the imposed boundary conditions. Typically, a weighted residual formulation is ap- 
plied, and these errors are orthogonalized with respect to a set of weighting functions 
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and minimized. In the FEM, typically a Galerkin-weighted procedure is applied, ex- 
panding the weighting functions in terms of the same locally defined shape functions 
as for the field variables. 

The FEM approach allows to generate models for the structural and acoustic prob- 
lem in both the time and the frequency domain with a total of n degrees of freedom. 
For the time-domain simulation, the resulting (semi-discretized) model is obtained 
as 


Mx + Cx + Kx =f, (3.10) 


with K € R”” being the stiffness matrix, C € R”” being the damping matrix, 
M e€ R”” being the mass matrix, and x and x representing respectively the first 
and second time derivatives of the nodal time-domain degrees of freedom x € R”, 
with external (time-domain) loads f € R”. 

In the frequency domain, the resulting discretized model becomes 


(K + jwC - w’M)x =f, (3.11) 


with x and fnow represented in the frequency domain. In general vibro-acoustic prob- 
lems, the model matrices K, C, and M can moreover be frequency-dependent as well. 
In the remainder of this chapter we will mainly focus on this frequency-dependent 
problem, as many of the time-domain aspects have been covered already in Chapter 2 
of this volume. Nevertheless, several additional points are raised, specifically focus- 
ing on converting these frequency-domain models into equivalent time-domain mod- 
els. 


3.1.3.2 Properties 


The discretization strategy of the FEM and the use of simple polynomial interpolation 
functions has its advantages and disadvantages. In practice this approach leads to the 
following general characteristics: 

Problem discretization and degrees of freedom The finite element approach di- 
vides the problem domain into a large number of small elements. The degrees of 
freedom in an finite element model are the nodal values of the field variables, and 
inside the elements, the dynamic field is approximated using simple polynomial 
shape functions. As frequencies of interest increase and wavelengths shorten, the 
finite element mesh needs to be refined to retain a sufficient accuracy as driven 
by interpolation and pollution errors [21, 34]. Practically, for linear elements, 
a rule of thumb is to apply at least 10 element per wavelength. Calculations at 
higher frequencies than considered by this rule of thumb for a given mesh can be 
considered erroneous and are physically not meaningful. 
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Problem geometric complexity Due to the fine discretization typically necessary to 
capture the wavelengths, the FEM has almost no restrictions regarding the geo- 
metrical complexity, as the elements are required to be small anyhow. 

System matrix properties In general, for undamped, viscously damped, or propor- 
tionally damped structures, the system matrices of the uncoupled acoustic and 
structural finite element model are real-valued, large, frequency-independent, 
symmetric, and sparsely populated with a banded structure. These properties 
allow for an efficient storage, solution, and reuse of the matrices for different 
frequencies [62]. 

Computational performance Although the finite element matrices are in general 
sparse and symmetric, because of the large number of finite element degrees of 
freedom, the solution of the finite element models is still computationally de- 
manding. The CPU time required to build and solve the system is proportional 
to nA’, with n being the number of degrees of freedom for the FEM and A being 
the bandwidth of the system matrix. 


3.1.3.3 Vibro-acoustic coupling 


The uncoupled acoustic and structural subproblem result in systems of equations of 
the format presented in equations (3.10)—(3.11), in which the primary variables for the 
acoustic domain are the nodal pressure vector p and for the structural domain the 
structural displacement vector u. By accounting for the coupling conditions between 
both domains, the following finite element system of equations is obtained in the fre- 
quency domain: 


Ce 3d F J il M, 0 |) [P] | 
+ jw -Ww . = 

0 K, 0 C, —PoK. M, p 

where the subscripts a, s, and c denote the uncoupled acoustic and structural system 


matrices and the coupling matrices, respectively. Equation (3.12) can be written more 
compactly as 


] G9 


f 


(Kup + JOC, s= w°M,,,)X up? (3.13) 


up 7 
where the subscript up refers to the use of the structural displacements and acous- 
tic pressure as primary field variables in the structural and acoustic subproblems, re- 
spectively. This system of equations has the same shape as the uncoupled structural 
dynamic and acoustic finite element models. The coupled system matrices are still 
sparse and frequency-independent [15]. It is worth noting that the coupled stiffness 
matrix K,,, and mass matrix M,,, are no longer symmetric due to the presence of the 
coupling matrix K,. 
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3.2 General comments on model order reduction for 
vibrations 


3.2.1 Choice of linear system solver 


Before we dive into model order reduction methods, a comment on the solution of a 
sparse linear system is in order. The linear system of equation (3.11) is usually hard 
to solve by an iterative method, even for low frequencies. The condition number of 
the frequency-dependent system matrix is usually high and the matrix is often far 
from definite. An additional difficulty is that classical preconditioners, such as multi- 
grid and incomplete factorizations, are often only reliable when specific mathemati- 
cal properties are satisfied, such as (positive) definiteness or the M-matrix property. 
The literature proposes techniques to overcome this difficulty by applying these pre- 
conditioners to another, modified system matrix that is more favorable to precondi- 
tioning. For undamped acoustics (C = 0), the incomplete factorization was applied 
to K + aM [33] with an optimal choice of a, which is currently known as the shifted 
Laplace preconditioner. For damped acoustics with nonzero C, one could apply the 
preconditioner to K + jwC + wC + 2 ju" M (e. g., see [36]). 

We now present numerical examples to illustrate properties of direct methods, 
i. e., methods based on a sparse lower-upper (LU) factorization, and preconditioned 
Krylov solvers. Consider the numerical example from [35] with the three-dimensional 
mesh shown in Figure 3.2a. The matrix K — wM is real symmetric, is of order 140,228, 
and has 1,822,668 nonzero elements. The LU factorization of K — w*M using MUMPS 
[40] on a Dell Latitude 6400 required 13 seconds in 2009. The construction of a Krylov 
basis of dimension 50 by the Lanczos method (excluding the LU factorization) required 
15 seconds. This shows that the factorization cost is highly dominant. Consider an- 
other example from [36], which is a finite element problem with spherical infinite el- 
ements. The mesh is given in Figure 3.2b. The system matrix in (3.11) is nonsymmetric 
and complex-valued. The order is 72,976 and the matrix contains 1,541,904 nonzero 
elements. Timings on a Dell Latitude 6400 in 2009 showed that the direct solve with 
MUMPS required 119.13 seconds (LU factorization and forward and backward solve). 
BiCGStab with the ILU preconditioner from [36] required 52.20 seconds at a frequency 
of 200 Hz and 103.71 seconds at 500 Hz. It is well known that iterative methods perform 
typically badly for higher frequencies, which is confirmed by this experiment. We also 
see that direct methods are competitive with iterative ones for this problem. 

In our experience, iterative methods are not competitive with direct methods for 
most applications in finite element (vibro-)acoustics. Therefore, the use of a direct 
linear solver is very common in model order reduction. For a direct linear solver, the 
highest cost is the sparse LU factorization. Once the LU factorization is performed, the 
solve cost is only a fraction, typically 10 % or less, of the factorization cost. For this rea- 
son, the methods based on Krylov and rational Krylov sequences are very popular and 
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a. [35] b. [36] 


Figure 3.2: Meshes for linear solver benchmarks. 


by far the most efficient methods in terms of computational cost, since they reduce the 
number of matrix factorizations. In practice, it implies that methods such as the dom- 
inant pole algorithm, the iterative rational Krylov algorithm (IRKA) [2], and variations 
on these methods are only used for models of relatively small scale. Unfortunately, 
Krylov methods usually lead to larger orders of the ROM for the same accuracy than 
balanced truncation or IRKA. 


3.2.2 Frequency limitation 


For model order reduction, one should be aware of the fact the mesh is only valid 
for a limited frequency range. This is another reason why (rational) Krylov methods 
are popular, as they focus on a limited interval on the ;w-axis. Sometimes, the high- 
frequency eigenvalues of the discrete model are the most dominant ones, so that de- 
fault implementations of methods such as the dominant pole algorithm, balanced 
truncation, and IRKA cannot be used. However, there are modifications of these meth- 
ods that limit the frequency range as well. For the dominant pole algorithm, it is suffi- 
cient to modify the definition of dominance, taking into account the frequency limita- 
tion [49]. For balanced truncation the frequency limitation can be taken into account: 
In [7], the right-hand side of the Lyapunov equation is modified in such a way that the 
high-frequency content is filtered out. However, the obtained ROM may no longer be 
stable. 


3.2.3 Modal approximation 


For the order reduction of mechanical models, modal approximation approaches 
have been very popular over the past decades both in research and engineering prac- 
tice. These approaches are based on the practical observation that the response of 
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a mechanical second-order system is typically dominated by its lowest-frequency 
modes. This is the result of the dynamics stiffness increasing considerably for higher- 
frequency modes, resulting in small-amplitude contributions to the overall response. 

In its most basic version this approach uses a set of free-free eigenmodes V of the 
undamped model for the reduced-order basis [24]: 


(-M,w; +K,)V;=0, Vo; < Wmax (3.14) 


where V; is the modal shape vector at pulsation w;. An important benefit of this ap- 
proach is the inherent inclusion of the limited frequency range in which the model 
discretization is valid. However, this basic approach tends to lead to poor accuracy 
as it does not account for the particular interface conditions of the model. In order to 
robustify with respect to these conditions, a range of extensions have been proposed 
which augment the initial modal basis with specific interface modes. The best-known 
reduced-order basis in this framework is the component mode synthesis approach 
[16]. However, over the years many authors have proposed a range of approaches 
which fit this framework [12]. 

A major reason why these approaches tend to be popular in practice is the clear 
physical interpretation of these ROMs. This if often important because practicing en- 
gineers tend to prefer approaches which they understand, as the ROM setup typically 
requires tuning for different applications. However, these methods also suffer from 
two important drawbacks: 

— Noreliable error estimators exist for modal approximation approaches, leading to 
tedious tuning by the user to achieve the desired accuracy. 
— The computation of the free-free modes can be expensive for-large scale models. 

This can limit the overall gains in the model evaluation time when also accounting 

for the MOR setup time. 


As a result of these drawbacks, they are not discussed in more detail in the remainder 
of this chapter. Nevertheless, these modal reduction approaches are still very popular 
in practice. They moreover serve as inspiration for a range of novel nonlinear model 
order reduction schemes like the modal derivative approach for nonlinear problems, 
as discussed in Chapter 2 of this volume. 


3.2.4 Rational Krylov methods 


For the sake of notation, let us repeat the idea of Krylov methods from [9, Chapter 3] 
and their mathematical and algorithmic properties important for this chapter. Con- 
sider the following linear (descriptor) state-space model in the Laplace variable: 


Ax — sEx =b, (3.15) 


H = cĪx, 
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where A, E € R™", b,c € R“ with ñ large. (Rational) Krylov methods build a basis of 
dimension k, which we denote by matrix V € C’**, for the column space of 


[K,»,((A - OE) 'E, (A - 0,E)'b),... Ky, (A - 0,E) ‘E, (A -0,E) 'b)], 
with K,,,(T, b) the order m Krylov matrix 
K„(T, b) = [b, Tb,..., 1b]. 


This sequence uses p different shifts, the i-th shift with multiplicity m;. Now let W € 
c™* be an arbitrary rank k matrix. The order k ROM 


Ax - sEX = b, (3.16) 
H=@'%, 
with A = WAV, E = WTEV «€ CK, and b = W’b, ¢ = Vc e CĂ, has the following 
moment ere PENS 
Ro)=- 2 Hop, j=0,...,.m;-1, i=1,...,p. 
If, in addition, the columns of W span the adjoint space 
[Km ((A - OE) TE", (A - 01E) ™™c),..., Km, ((A - 0E) TE", (A - 0E) “c)], 

the ROM (3.16) has the following moment matching properties: 


d . 
ds! 
The advantage of Krylov methods is that the poles can be chosen so that 
1. the limitation to a frequency range is respected; and 
2. the number of large-scale LU factorizations is small (in this case p). 


d 
agi h(i) = 5 agit i» j=0,...,2m;-1, i=1...,p. 


The downside of the IRKA [2] and the dominant pole algorithm [46] is the large compu- 
tational cost due to a large number of sparse LU factorizations. In fact, if the number 
of factorizations is as large as the number of points required by a POD approach for 
the frequency axis, there is no interest in using such methods. Greedy methods (see 
[10]) do not guarantee the same approximation error as Hœ- and H,-minimization of 
the error, but they require fewer matrix factorizations. 

The main disadvantage of a Krylov method is that, in general, stability is not guar- 
anteed. We will see in Section 3.5 an application for which stability is guaranteed. An- 
other disadvantage is that the size of the reduced model may not be minimal for the 
same accuracy as balanced truncation. Krylov methods are popular because they pro- 
duce a reasonably good reduction in a relatively low computational time for the ROM 
setup. As an extension, the obtained ROMs can be further reduced by balanced trun- 
cation, e. g., [26], in order to mitigate the high setup cost of the balanced truncation 
model on a high-order model. We give other examples in Section 3.4.4. 
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3.2.5 Concept of linearization 


The standard approach for mapping the second-order matrix polynomial in (3.11) to 
a linear form is called linearization. Linearization allows to use model order reduc- 
tion methods for linear models as in (3.15). Best known is the companion linearization. 
It doubles the dimension of the state space, but the system is linear and produces 
the same transfer function as the system with quadratic frequency dependency. Sys- 
tem (3.11), with output H = c’ x, can be rewritten as the linear model 


K 0 Cc M x f 
(Lo) Lol) Led lol ean 
T 
aee aa 
O] Ljwx 

Any method for linear systems can be used for reducing (3.17). It should be noted that 
the state vector now has two components: x and jwx. The Krylov vectors now have 
length n = 2n. However, due to the structure of the state space the memory cost of the 

iteration vectors can be halved [3]. 
Linearizations are also used for other polynomial, rational, or even fully nonlinear 
dependencies on the frequency. Efficient implementations of Krylov methods (one- 


and two-sided) rely on a similar property as the state vector of second-order problems 
[53, 30]. We will give more examples in Section 3.4. 


3.3 Structure-preserving model order reduction 


In many cases, it may be important to have a ROM that respects the structure of the 
large-scale model. Such structures can take various forms: real matrices, symmet- 
ric matrices, polynomial frequency dependence, and so on. For example, the typical 
structure for frequency-domain vibration analysis is the form of (3.11). Finding a ROM 
of exactly the same structure (symmetric matrices, quadratic frequency dependence) 
may be beneficial for keeping spectral properties, e. g., but also to physically interpret 
the ROM. 


3.3.1 Quadratic frequency-domain structure 


In this section, we discuss the exploitation of quadratic structure as in (3.11). The 
choice of ROM depends on its purpose. If the model is to be coupled with other models 
in the time domain, a linear model is usually preferred, since the connection with the 
time domain is straightforward. For reliable time-domain simulation a stable model 
(which mechanical systems inherently are) is required, but this is not always the case 
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for the ROM. By respecting the quadratic structure, the stability can sometimes be 
guaranteed. 

For second-order systems (3.11), Krylov methods rely on linearization (3.17). As- 
sume that the reduced model is obtained by projection of (3.17) on subspace V spanned 
by the columns of the full-rank matrix V € C2“. Decompose 


V xk 
V= Kal V,V,¢C™. 


Let V be computed by an interpolatory model order reduction method, i. e., a (rational) 
Krylov method, on (3.17). Interpolation at w = w; implies that x(w;) is spanned by the 
columns of V, and jw;x(w;) by the columns of V3. This suggests that there is a strong 
connection between V, and V,. Indeed, second-order Arnoldi (SOAR) [3] and two-level 
orthogonal Arnoldi [31], for example, exploit this property and express V; = QU,, i = 
1,2, with Q € Cc”, U; € co , where £ < k. Instead of building a linear reduced model, 
one might, as in SOAR, compute a second-order reduced model by projection on the 
column span of Q: 


(R+ jw? — w°M)x =F (3.18) 


with K = Q’KQ, C = Q'CQ, M = QMQ, f = Q'f, andé = Q’c. 
The situation is more complicated for two-sided model order reduction. Indeed, 
the state vector of the adjoint of the linear problem 


K’ +jwC” -jol Z c 
= > .1 
joM I JL | | | Ge) 


#=[0) (some) 


has the two components z € C” and —jwM’z e€ C”. Now assume that the reduced 
model is obtained by projection on subspace W spanned by the columns of the full- 
rank matrix W € C7”, Decompose 


W, 


W= h | , W,w,.c™. 


2 


Let W be computed by an interpolatory method on (3.19), i. e., there are w so that z(w) 
is spanned by the columns of W} and jwM’z(w) by the columns of W. This suggests 
that there is also a strong connection between W, and W,. Indeed, in [30] this property 
is exploited by expressing W, = ZT, and W, = M’ZT, with Z € c™*, T; € C™*, where 
é<k. 
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A linear ROM is obtained by projecting (3.17) on the right by V and the left by W7. 
A structure-preserving alternative is to project (3.17) by 


[o al Do zul 

o Qo [o0 Z™’ 

on the right and the left, respectively. Note that the first spans the columns of V and 
the second the rows of W7, so the interpolation properties are preserved with these 
bases. This leads to 


bares | x |- H 
-juM M | ljox! lol 


a-f] L 


with K = ZĪKQ, C = Z’CQ, M = Z'MQ, f = Z’f, and é = Q’c. Assuming that M has 
full rank, this is equivalent to the frequency-nonlinear model (3.18). 

We now illustrate one-sided linear and second-order reduced models for the 
following example, whose description is taken from [60]. The model describes a foot- 
bridge located over the Dijle river in Mechelen (Belgium). It is about 31.354 m in length 
and four tuned mass dampers (TMDs) are located at nodes at 11.299 m, 19.314m, 
10.549 m, and 20.309 m, respectively, each of which weighs 40.72 kg. The discretized 
model is 


4 
(x, + jwCy +) K; - 0M) =f, 
i=l 

where Ky and M, are obtained from a finite element model with 25,962 degrees of free- 
dom, as shown in Figure 3.3a. Here, Co = 0.1003M, + 0.0001591K, represents Rayleigh 
damping, K; is a matrix with four nonzero entries that represents the interaction be- 
tween the i-th TMD and the footbridge, and the input vector f represents an excitation 
equally spread among the locations of the TMDs. All matrices are symmetric positive 
semi-definite. We used 20 iterations of Arnoldi’s method (Krylov method) with single 
shift 50.5 on (3.17). A linear ROM was obtained by one-sided projection A = V* AV and 
B = V*BV anda second-order model of the form (3.18). Figure 3.3b shows the resid- 
ual norm of (3.11) for w € [0,100] when (3.17) is projected on the Krylov space (linear 
model) and when (3.11) is projected on the column range of Q (second-order model). It 
is easily seen that around the interpolation point, the error is of the same order of mag- 
nitude but that further away, the second-order model has lower residual norms. This 
can be explained by the fact that projection of (3.11) is equivalent to projecting (3.17) 
on a larger subspace, similarly to two-sided models. An additional advantage for the 
second-order model is that the matrices in (3.18) are Hermitian semi-positive definite, 
so that stability of the reduced model is guaranteed. This is not the case for the linear 
ROM. 
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Figure 3.3: (a) Mesh. (b) Residual norm on (3.11) for the linearized model and the original second- 
order model. 


3.3.2 Lagrangian structure in mechanical models 


For classical mechanical systems, the equations of motion in the form as presented in 
equation (3.10) can be obtained from a Lagrangian description of the system, with the 
Lagrangian £: 


L=K-V+w (3.20) 
- İ4TMx — Lx? kx xf, 
2 2 


where K, V, and W are respectively the kinetic energy, the internal elastic energy, 
and external work. The equations of motion are then obtained by applying Hamilton’s 
principle to the Lagrangian: 


d (ðL ðL 
R E (3.21) 
This Lagrangian structure inherently embeds energy-preserving behavior in the sys- 
tem, ensuring stability for long-term simulations. It is therefore important that the 
ROMs for mechanical systems respect this structure. In order to ensure that the re- 
sulting ROM equations of motion comply with this Lagrangian structure, a direct sub- 
stitution in the Lagrangian can be performed. For a constant reduced-order basis, one 
can substitute for the reduced degrees of freedom q: 


x=Vq, x~V4q, (3.22) 


which results in the Lagrangian expressed as a function of the reduced-order degrees 
of freedom q: 


L= 5a'v™Mvq = 5a'V"KVq _q'v'f, (3.23) 
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such that after application of Hamilton’s principle for the reduced degrees of freedom, 
the reduced equations of motion become 


V'MVq+ V'KVq = V'f. (3.24) 


As this form results from the (reduced) Lagrangian of the system, it inherently pre- 
serves the stability of the underlying mechanical model. 

This implies some care on how the model order reduction on a mechanical sys- 
tem is applied, as the models are often presented in different form. For example, for 
practical time-domain simulations, equation (3.10) is often converted to a first-order 


state-space model as 
x 0 I] [x 0 
lel = Lee ol fel lel: mee) 


In a regular projection framework, state-space system (3.25) would be reduced as 


H =Vq= 4 qa V'v=!I (3.26) 


and one would obtain a reduced system of equations as 


0 Vi mal | 0 
i 3.27 
a eee o |1* [vif Ba) 


It is now clear that this system of equations cannot be converted back into a La- 
grangian form like (3.23), such that time-stable and energy-preserving behavior is not 
naturally preserved. From this perspective, it is advisable to always apply symmet- 
ric projection of the second-order system for mechanical models if the ROM will be 
employed in time-domain simulations. This again shows the conceptual benefit of 
approaching mechanical ROMs through a one-sided projection on the second-order 
system. 


3.4 Complex frequency dependencies 


3.4.1 Sources of nonlinearities 


To mitigate noise and vibration issues in mechanical engineering applications, damp- 
ing treatments are often applied. The physical behavior of these treatments are mostly 
represented by (complex) frequency-dependent behavior. Poro-elastic materials, for 
example, are often used in a sound absorption context. An overview of modeling ap- 
proaches for poro-elastic materials can be found in amongst others [1, 17]. Viscoelastic 
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materials are often applied in a constrained component setting as this ensures shear, 
and thus more dissipation in the sample. Their behavior strongly depends on temper- 
ature and frequency. Detailed descriptions of various mathematical models are given 
in [20, 45]. 

When unbounded acoustic domains are considered (exterior problems), using fi- 
nite element models, the domain has to be truncated in practice. In this case the Som- 
merfeld radiation condition [14], which ensures that no acoustic energy reflects back 
from infinity, has to be approximated. Commonly applied approaches, like absorbing 
boundary conditions [4] and perfectly matched layers [11], also result in complex fre- 
quency dependencies. 


3.4.2 Rational and polynomial frequency dependencies 


We assume that the frequency-dependent full-order system, of order n, can be written 
as 


A(s)x =f, (3.28) 
H =c’x, 
d-1 
with A(s)= ) ¢;:(A;+sB;) and s = jw. 
i=0 


We introduce the following notation: 
A + sB = [Ap +SBo,...,AMg-1 + SBg_1], 
® = [$o,..., al, 
where ® forms a set of polynomials or rational polynomials that satisfy 
(P+sR)® = 0, 
with P, R € C44 constant matrices. This leads to 
A(S) = (A+ sB)(® @I,,). 


As a result (3.28) can be rewritten as an order nd linear system: 


PoxX f 
A+sB px \ | 0 
acne a) eat A 
Pq-1X 0 
Pox 
H=[c'0--- 0] 
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or in compact form 


| A + sB 


(P+sR)@ “| (DW @x) =e, ef, (3.29) 


H =(e,@c)'(W@x), 


which is clearly linear in s. For the companion linearization from Section 3.2.5 of 
second-order problems, P = [-1,0], R = [0,1], ®(s) = [1,s]’. For the rational matrix 
polynomial 


1 


A(s) = (Ag + SBo) oF s_ 


1 
A, + SB,) + — (A; + sB»), 
7, (hi + 8B) + 5 (Aa + sB) 
a possible linearization is 


Ao +SBy A,+SB, A,+SB, x 
I Ool- sI 0 x |= 


S-00 


I O 0,1 - sI =a 


O O m 


In this case, the choice of basis and the choice of linearization are important. Some 
formulations lead to a descriptor system, which may require additional care for proper 
setup of the ROM. However, for models in the frequency domain, we have never expe- 
rienced difficulties in this case. 

It can be proven that the (rational) Arnoldi method for (3.29) produces iteration 
vectors that take the following form: If the k vectors are collected in an ndxk matrix Vy, 
this matrix can be factored as 


Ve = (1g2QU;, Q€ C”, U e CK, (3.30) 


with £ < k+d [31, 53]. The advantage of this factorization is that large-scale operations 
only happen with columns of Q and that all other operations are small-scale. This 
leads to a reduction of the full unstructured storage of Krylov vectors of order ndk to 
at most n£ + £dk with £ < k + d [53]. For two-sided Krylov methods, the exploitation of 
the structure of (3.29) is possible, but is more involved [42, 22, 30]. In each iteration of 
a Krylov method, a linear system is solved. In [53], it is shown that this requires matrix 
vector multiplications with A; and B; and a linear solve with A(o;) at step k. 

As for second-order problems, Q can be used to develop a nonlinear frequency- 
dependent reduced model: 


d-1 
A= > (Aj + SBi)o;, A, + sB; = Q* (Aj + sB;)Q, j= 0,...,d-1. 
j=0 
If all shifts o,,...,0, are distinct, then 


Range(Q) = Range([x(o,),...,X(0;,)]) = Range({A(o,) ‘£,...,A(o,) “f]). (3.3) 
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When the shifts are all equal, 
Range(Q) = Range([x(0,), dx(0,)/ds,..., d‘1x(0,)/dsk}). (3.32) 


In other words, the range of Q spans the moments of the state vector x of (3.28). If 
only Q is required, a linearization is not useful, since Q can be computed directly 
using (3.31) or (3.32), which requires linear solves with A(s). However, in the case of 
higher-order interpolation, as in (3.32), the computation of the derivatives is equally 
complicated as performing a Krylov step with the linearization. 

If a linear ROM is required, then a linearization is useful. In this case, (3.29) is 
projected using the full Krylov vectors given by the factorization (3.30). For two-sided 
model order reduction, with simple interpolation points, a linear reduced model can 
equally be obtained by putting samples of H from (3.28) in a Loewner matrix; see [9, 
Chapter 6]. In this case, the reduced model is defined with 


(0;)-H(1%) H(0,)-H(1) 
01-7 Ok-Ti 
E= : : > 
H(01)-H (Tx) H(0o)-H (Tx) 
O1-Tx Ok-Tk 
0,H(0,)-T,H(T,) OH (0x)-TH(11) 
01-7 Ok-Ti 
A = . . 
0,H(0,)-T,H(T,) OH (0,)-T, A(T) 
01-T On-Tk 
and 
H(0;) HCT) 
-| Ho) | | A) 
b= . C= ; 
(0x) H(t,) 


When the interpolation points are not too close to each other, this is a reliable and 
easy-to-implement method. 

We now present two cases of nonlinear frequency dependency. The first case, pre- 
sented in Section 3.4.3, uses (3.31) to directly construct Q and computes a linear ROM 
using Loewner matrices. The second case uses (3.32) to reduce the number of matrix 
factorizations; see Section 3.4.4. 


3.4.3 Matrix-free model order reduction for vibro-acoustic 
systems with complex noise control treatments 


Many model order reduction methods cannot straightforwardly cope with frequency 
dependencies in vibro-acoustic models, resulting from, amongst others, complex 
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damping treatments or infinite acoustic domains. In this section, a matrix-free Krylov 
MOR method [27] is presented that does not require knowledge on the underlying 
mathematical model and that can straightforwardly handle problems with frequency- 
dependent parameters in exactly the same manner as problems with constant prop- 
erties. The method only works in the frequency domain and is based on a two-sided 
model order reduction approach with distinct interpolation points using the Loewner 
matrices as discussed in Section 3.4.2. The frequency-dependent transfer function 
between input and output is indicated by H(w). The use of the Loewner matri- 
ces leads to a new approximative transfer function H which interpolates the origi- 
nal transfer function H, using the forced responses at the 2k distinct interpolation 
points. 


3.4.3.1 Frequency selection and convergence evaluation 


In the following, let H;, denote the transfer function (or frequency response function) 
obtained using a Loewner model of order k, i. e., using k right interpolation points and 
k left interpolation points. Jonckheere et al. [47, 48] present two criteria to assess the 
convergence of the ROM, which leads to a greedy method for determining the inter- 
polation points. The first convergence criterion is based on the relative error between 
two subsequent ROM approximations, which is computationally cheap to evaluate: 


fw) - Âp (w) 


x (3.33) 
Ay (w) 


EROM,ROM = Max 


However, the relative error between two subsequent approximations may be low in 
case the newly added frequencies are close to the frequencies added in the previ- 
ous iteration and bring limited additional information to the interpolation. Therefore, 
a second convergence criterion is embedded by comparing the (approximated) trans- 
fer function obtained using the current ROM with the (exact) transfer function, calcu- 
lated by solving the full system, at the newly selected frequency lines w; which would 
be needed to build a new ROM in the next iteration: 


Hw) - A,(w) 


= (3.34) 
H,(w) 


EROM,Full = MAX 


The relative error between two subsequent ROMs (3.33) is used to select new frequency 
lines to enrich the ROM in the next iteration. This selection is done in a cascading 
way: (i) the frequency at which the maximum error occurs is selected, (ii) the direct 
vicinity of the new frequency line is masked to avoid selecting neighboring maxima, 
thus to spread out the new frequency lines a bit, and (iii) from the remaining frequen- 
cies the one corresponding to the new maximum error is selected, and the procedure 
continues along (ii)—(iii) until the requested number of additional frequency lines is 
reached. 


96 — E. Deckers etal. 


3.4.3.2 Calculation example 


One example considering a mechanical system with complex damping treatments 
taken from [47] is presented. The example geometry is shown in Figure 3.4, which is 
taken from the Comsol manual [39]. This interior car cavity has dimensions of approx- 
imately 4.5m by 1.5m by 2m and has 26 acoustic modes below 300 Hz. All acoustic 
boundaries are rigid, except for the fire wall (indicated in light blue), where an acous- 
tic acceleration of 1 m/s? is imposed. The density of air is specified as 1.2kg/m? and 
the speed of sound as 343.8 m/s. On the seats a frequency-dependent impedance is 
applied, where the impedance is calculated from tabulated values of absorption co- 
efficients. The acoustic pressure is tracked at the driver’s ear as output quantity. The 
finite element model consists of 1,372,332 degrees of freedom. The calculation of a sin- 
gle frequency line took approximately 69.5 s for the damped case on a Linux Cluster, 
using two 10-core Ivy Bridge Xeon E5-2680v2 CPUs (2.8 GHz, 128 GB RAM). The fre- 
quency range from 1 Hz to 300 Hz is simulated with a 1 Hz step to set up the reference 
data. 


Figure 3.4: Car cavity geometry with porous seats with complex frequency-dependent behavior. 


For this example, the starting point is a steady-state dynamic equation of the form 
[K(w) - w’M(w)]x = f, (3.35) 


where M, K € R™” are in this case frequency-dependent complex mass and stiffness 
matrices accounting for damping effects. Note that in this case s = ( jw)’, in contrast 
to Section 3.4.2. 

Figure 3.5 shows the original and fitted frequency response functions. The matrix- 
free approach starts with an initial calculation of two frequency lines, setting the 
boundaries of the frequency range of interest (5 Hz and 300 Hz). Thereafter, two addi- 
tional frequency lines are computed and used to build a new ROM. When the requested 
accuracy of 1% is met, both on €goy,rom and On Egoy ru, the algorithm terminates 
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Figure 3.5: Acoustic frequency response function for car cavity with porous seats and final approxi- 
mation for the acceleration transfer function; crosses indicate the interpolation frequencies. 


(after 24 iterations, requiring only 48 frequency evaluations). A speedup factor of 6 is 
obtained for the acoustic frequency response assessment in this particular case. 


3.4.4 Rational approximation 


When A(s) is generally nonlinear, i. e., not polynomial or rational, linearizations can 
still be used on polynomial or rational approximations. Rational approximation, in 
particular, is very powerful, as we will now illustrate. The idea presented here is to 
approximate A(s) by a rational polynomial with d terms as in (3.28). Over the years, 
a number of approaches have been developed to perform these approximations. In 
[59], a Taylor expansion is used. In [52] and [50], A(s) is approximated by a truncated 
Padé series. In [50], windowing is used to cover the entire frequency range, and in [38], 
a spectral discretization is used. In [29], a rational approximation based on the adap- 
tive Antoulas—Anderson method, also known as AAA and pronounced as “Triple A,” 
is proposed. 

Here, we will present the idea from [29], since it is an elegant and user-friendly 
way to find a rational approximation for the model. For the implementation details, 
we refer to [29]. Assume that A has the following form: 


m 
A(s)=K+sC+s'M+ ) (A, +SB,)f,(5), 
p=1 
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with fp : C > C a scalar function in s. First, the general nonlinearity is approximated 
by a rational function, expressed in barycentric rational form [41]. After the approxi- 
mation, the matrix can be written as the rational matrix polynomial 


Fr w;/(gw - &) 
io w/ (Jw - ay 


where w;, i = 0,...,d — 1, are oe, the weights and ¢; the support points. The AAA 
method chooses d, w;, & fori = .,d—1so that 


d-1/ m 
A(s)=K+sC+s°M+) ( Y (Ay + SBy fy (GD 


i=0 \p=1 


sup oe Or eee (3.36) 


WE [Omin Wmax Eia w/w = &) 
is below a given tolerance for p = 1,...,m. The AAA method determines the parame- 
ters automatically in very little time. Only the function values of f, for the test set are 
required. There is no need to compute derivatives. The criterion (3.36) is discretized 
in a number of points on the frequency axis. The support points ¢; are chosen using a 
greedy method. The minimization of the residual leads to the weights w;. 

The following example is based on a system from [25, 37]. The results are reported 
n [28]. For this example, we consider the following time delay differential equation: 


3 
~ = Vv +) av + bu, 
j=1 
q = (c, v). 


We set the domain to [0, 7r] x [0,7r] x [0,7], and let 


a(x, y, Z) = 2sin(z — x) sin(y) sin(z), 
a(x, y, Z) = 2sin(x) sin(x - y) sin(z), 


a3(x, y, Z) = 2sin(x) sin(y) sin(7 — z), 


and T, = 1, T3 = 2, T3 = 4. For the domain discretization we use central differences with 
N discretization points in each spatial variable. After transforming to the frequency 
domain, we get 


| (Ao -SI + Èi Aje ‘°)x(s) = —bu(s), a 


H(s) = c*x(s), 


where s = jw =, Aj € R", x,b,c € R”, and n = N’. We are interested in solutions in 
the range w € [0.01,10]. The resulting system matrices are symmetric. Furthermore, 
the input application vector b is chosen as a vector containing random values in [0, 1] 
and c = b. In the following, we show the relative error of the transfer function: 


|H(s) - H(s)| 


e6) = TE 


> 
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with 
H(s) =c*A(s)'b, and HA(s) =¢*(A-sE)'b. 


The error is computed over the test set w;, j = 1,...,500, with logarithmically spaced 
points in the interval [10~’, 10], as values closer to zero are of particular interest. 

For the AAA approximation, a test set of size 10,000 is chosen over the interval 
[0.01, 10]. The AAA algorithm results in an approximation of the three nonlinear func- 
tions with d = 22. 

We now compare the results of three numerical methods: 

RKS: A rational Krylov sequence of order K = 150 obtained by using the three shifts 
0), 57,10), each one 50 times. This leads to a linear reduced model. 

LOW: A linear reduced model using Loewner matrices is used of order k = 40 with 
equidistant interpolation points in [0.01), 104]. 

HYB: A hybrid approach where the reduced model of order K = 150 from RKS is 
further reduced to order k = 40 by applying LOW. 


For N = 4O, i.e., n = 64,000, the relative error for RKS of size K = 150, for LOW 
for k = 40, and for HYB is plotted in Figure 3.6. From the figure, it is clear that the 
Loewner pencil leads to a lower reduced dimension for the same error level in the 
higher-frequency range. The hybrid approach produces a model of size 40 with an 
error that is the maximum of the errors of RKS and LOW. The difference between the 
Loewner and hybrid approaches lies in the execution time. For the Loewner approach, 
the simple interpolation using Loewner matrices required 80 large sparse matrix fac- 
torizations, where for the rational Krylov and hybrid approaches only three matrix 
factorizations were performed. The cost of the Loewner approximation on the model 
of order 150 in the hybrid approach is therefore negligible. The computations were 
timed and averaged over three runs, on a machine with 64-bit Intel processor, 28 cores, 
2.6 GHz processors, and 128 GB RAM. The hybrid approach required 384.7 seconds, 
where the Loewner approach required 1,814.9 seconds, which is a significant factor of 
4.7 of the computation time for the hybrid approach. 

A second example is described in [29]. This model was generated using a mesh 
from Siemens Industry Software and applied poro-elastic material properties. The fol- 
lowing matrix-valued function describes the nonlinear behavior of the sound pressure 
inside a car cavity with porous seats, represented by a Johnson—Champoux-—Allard 
equivalent fluid model [1]: 


A(w) = Ky + hy(w)K, - w (Mo + hy (w)M,), 
where Ko, K,;,My,M, € R™" with n = 15,036 are symmetric, positive semi-definite 


matrices and w is the angular frequency. The nonlinear functions are given by 


of |. 4a2n 
hy(w) = 2 a(w) = A + ooh + WP PRE? 
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Figure 3.6: Relative error as a function of the frequency f = Re(s) for n = 30. Linear pencil of reduced 
dimension K = 150, Loewner with k = 40, and IRKA applied to the reduced pencil with k = 40. 


and 


-1 8 ; A 
hy(w) = o(y- n7 } a'w) = 1+ iy y+ iwpo 


with the parameters defined in Table 3.1. 


Table 3.1: Constants of the car cavity model. 


Qo 1.7 o 13500 kg m`? s~! $ 0.98 


n 1.839: 107° A 80-10 m N 160-107 m 
y 1.4 Po 1.213 P, 0.7217 


The AAA approximation was built by approximating hg and hy in the w range 
[50, 10007]. This led to a rational approximation with 12 support points with a rel- 
ative error of 10°” for both functions on this interval. Six shifts, namely, 1595.8), 
2173.92, 2742.78 7, 1125.87), 3039.57, and 50), were selected using a greedy approach 
on the linear system’s residual norm, where 20 Krylov iterations were performed for 
each shift. The dimension 120 was further reduced by applying POD in the frequency 
domain on this model to obtain an order of 77 for a relative tolerance of 10°°. 


3.5 Vibro-acoustic model order reduction 
3.5.1 Stability-preserving model order reduction for 
vibro-acoustics 


For coupled vibro-acoustic problems, the advantages of MOR have been demon- 
strated in the literature in the frequency domain (see, e. g., [23, 44]). However, popular 
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reduced-order techniques for frequency-domain analysis cannot directly be applied 
to time-domain analysis as these techniques do not necessarily preserve the stability 
of the original system [8]. While this loss of stability has no negative consequences on 
the frequency-domain analysis of the vibro-acoustic system, it may lead to a diverging 
response in the time domain. 


3.5.1.1 Conditions for stability of a linear descriptor system in a one-sided 
projection 


In [54] general conditions are derived that ensure to preserve the stability of a linear 
descriptor system in a one-sided projection. The semi-discretized system model in the 
time domain 


Ex(t) = Ax(t) + bu(t), 
H(t) =e’ x(t) (3.38) 


is critically stable if all of the generalized eigenvalues of the matrix pair {A, E} havea 
negative real part [18]. It is proven that stability is preserved in the one-sided projection 
of the descriptor system in (3.38) if E = Ef and if E is positive definite and if A is 
negative semi-definite. A matrix Q € R™” is positive semi-definite if 


x’ Qx >0 


for any nonzero vector x € R". 


3.5.1.2 u-p formulation and modified u-ġ formulation 


The vibro-acoustic model is generally described by (3.12), but depending on the ex- 
act formulation used for the model, different coupling matrices are present. Through 
a Galerkin approach for the finite element model for the acoustic problem, it follows 
that the acoustic system matrices K,, M,, and C, are all symmetric positive (semi-)def- 
inite. Typically, M, is of full rank, which makes it strictly positive definite. Whether K, 
is strictly or semi-positive definite depends on the boundary conditions. Similar con- 
clusions can be drawn for the structural finite element model and the corresponding 
system matrices K,, M,, and C,. 

In the case K, and K, are positive semi-definite, the zero eigenvalues of these ma- 
trices correspond to rigid body modes of the system, manifesting themselves as sys- 
tem poles at the origin of the complex plane. These poles do not adversely affect the 
stability of the system and are disregarded in the subsequent analysis without loss 
of generality. This implies that we can consider K, and K, to be strictly positive defi- 
nite. 
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In [54] the definiteness of the global system matrices of the coupled vibro-acoustic 
finite element model equation (3.13) are investigated. While the coupled damping ma- 
trix Cp is positive (semi-)definite, K,,, and M,p are generally not positive definite due 
to the presence of K, and M,. Consequently, also E,,, and A,,, of the equivalent de- 
scriptor system (3.38) are indefinite. Even though the vibro-acoustic system itself is 
stable, this stability is possibly lost in a regular one-sided projection. 

Everstine [19] proposes an alternative, symmetric formulation for the vibro- 
acoustic problem which uses the scalar fluid velocity potential @ instead of the 
pressure p to describe the state of the fluid part of the vibro-acoustic system. This 
fluid velocity potential is defined by 


p =-p@, (3.39) 


with fluid density p, such that the vector containing the nodal pressure values in the 
fluid can be expressed as p = -pq@. The use of this u-@ formulation results in the 
symmetric system of equations which can be constructed from the u-p model matrices: 


M,,pXug + Ci gXup F KugXuġ = fup» (3.40) 


M o C, -oK K, 0 
M = S p = S C : K 2 S ; 
up E EN Cup px se) ud E pa 
f 
fo = | | (3.41) 
p > 
Ui fy 


where fs = f,. Note that also for E,4 and A,,g, the matrices of the equivalent descriptor 
system are indefinite such that stability is not preserved in a one-sided projection. 

It is shown in [54] that by changing the sign of the set of equations governing the 
acoustic degrees of freedom, resulting in the so-called modified u-@, leads to global 
symmetric positive definite Mnu and Kmug and positive semi-definite Cmug, ensuring 
that the conditions for stability for the equivalent descriptor formulation are fulfilled. 
Even though the modified u-@ formulation lacks symmetry as compared to the stan- 
dard u-@ formulation, a one-sided projection-based MOR preserves stability and the 
resulting ROMs are well suited for time-domain simulation. 


3.5.1.3 Extended projection basis-preserving stability using the u-p formulation 


As the poles of the u-p formulation and the modified u-@ formulation are equal, as 
shown in [54], stability should also be preserved when converting a system from the 
u-p formulation to the u-¢ formulation and vice versa. In order to do so, the block- 
partitioned structure is required, which gets lost in the reduction process. By carefully 
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selecting the structure of the projection basis, the block structure within the second- 
order system matrices can be retained, allowing for this conversion between both 
formulations. 

Let the regular projection basis V € C*™*, obtained through any one-sided 
projection-based MOR method (e. g., Krylov subspace projection, modal truncation, 
...), be given by 


nxk 


v- K : (3.42) 


with V, € C”*K the part of the projection matrix that corresponds to the structural de- 
grees of freedom and V, € c”™** the part corresponding to the acoustic degrees of free- 
dom. By augmenting this projection basis with zero blocks, an extended projection V 


= [V, 0 
Ve | ral (3.43) 
is obtained that achieves block structure-preserving MOR for coupled systems, similar 
to the approach presented in [6]. The space spanned by the columns of V contains 
the space spanned by the columns of V, such that when using V the reduced system 
will be at least as accurate as when using V. For practical implementation it is recom- 
mended to orthogonalize V, which equates to orthogonalizing both V, and V,. Also 
note that V has twice as many columns as V such that the projected system will con- 
tain twice as many degrees of freedom. Since V, and V, may be rank-deficient this can 
be slightly ameliorated by using a rank-revealing algorithm for the orthogonalization 
of V, and V,, but in our experience the total number of columns in V still remains 
close to twice the number of columns in V. 


3.5.1.4 Work flow 


Both approaches, starting from the modified u-¢ formulation and the u-p formulation, 
are summarized in Figure 3.7. 

It is advisable to use the modified u-¢ formulation for stability-preserving MOR of 
coupled vibro-acoustic finite element models. Keeping the system in u-p formulation 
necessitates the use of an extended projection basis V, resulting in a ROM with possi- 
bly up to twice as many degrees of freedom. This ROM approach was recently extended 
towards time-stable coupled exterior vibro-acoustic finite element simulations [55]. 


3.5.1.5 Calculation example — car interior with vibrating roof 


As an example, the vibro-acoustic behavior of a car interior is studied. The roof of 
the car is modeled as a flexible steel panel (E = 200GPa, v = 0.3, p = 8000 kg/m’, 
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Figure 3.7: Work flow for stability preserving vibro-acoustic model order reduction [54]. 


Projection onto V 


and Rayleigh damping [C, = aM, + BK,] with a = 10 and £ = 1- 10°’) with a thick 
ness of 2mm which is clamped at its boundaries and the interior is filled with air 
(9 = 1.225 kg/m, c = 340m/s). Linear elements are used with a resolution of at 
least six elements per wavelength up to 200 Hz. The resulting finite element mesh is 
shown in Figure 3.8. Near the plate the mesh is more refined since the wavelength of 
the bending waves in the structure is smaller than the acoustic wavelength at 200 Hz. 
A normal impedance boundary condition of twice the characteristic impedance of air 
(Zn = 2-1.225- 340 Pa s/m) is imposed on the surfaces of the seats in the interior cavity. 
All other boundaries of the acoustic domain are considered rigid (v,, = 0 m/s). A point 
force at (1.50, 0.08, 1.09) m excites the structure and the sound pressure is calculated 
at (0.73, 0.23, 0.68) m. 


Figure 3.8: Finite element mesh of the car interior geometry [54]. 
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The finite element model of the roof and acoustic cavity consist of 7.455 and 6,583 de- 
grees of freedom, respectively. A one-sided Krylov subspace projection was con- 
structed using SOAR [3, 58]. All ROMs consist of 60 degrees of freedom, except for 
the u-p projection with extended basis (equation (3.43)), which results in 120 degrees 
of freedom. 

Figures 3.9 and 3.10 show the frequency response functions and the relative error 
with the full model, obtained applying model order reduction on each of the models. 
All ROMs are able to accurately describe the full system behavior in the frequency 
domain up to about 200 Hz. The u-p projected system with the extended basis is more 
accurate than the ROM that is obtained with the nonextended basis. Figure 3.11 shows 
the location of the poles of the ROMs in the complex plane. Both the u-p projection 
without extended basis and the projection of the system in standard u-@ formulation 
lead to unstable ROMs. 


FRFs of the full and reduced-order models 
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Figure 3.9: Frequency response function of the car cavity model calculated with the full-order model 
and the reduced-order models. 


Finally, Figure 3.12 shows the convergence of the H,-norm of the relative frequency re- 
sponse function amplitude error £ of the different reduction methods, evaluated over 
the range of 1 Hz to 200 Hz. The method using an extended projection basis performs 
substantially worse than the other methods when comparing them in terms of accu- 
racy per degree of freedom. 
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Figure 3.10: Relative error on the amplitude of the frequency response function of the reduced-order 
models of the car cavity. 
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Figure 3.11: Poles of the different ROMs of the car cavity models. The nonextended basis u-p projec- 
tion and standard u-@ projection result in unstable time-domain behavior. 


3.6 Conclusions 


We presented an overview of methods for model order reduction of dynamical systems 
arising in (coupled) acoustics and vibrations problems encountered in mechanical 
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Figure 3.12: Efficiency of the MOR methods in terms of accuracy per degree of freedom for the car 
cavity vibro-acoustic model. 


engineering. The choice of methods took into account the range of relevant require- 
ments in practical engineering problems: computational efficiency, frequency limita- 
tion, stability, nonlinearity in the frequency variable, and preservation of underlying 
model structure. 

For computational efficiency, the number of large sparse LU factorizations should 
be small. This can be achieved by using Krylov methods with a greedy selection of 
the interpolation points and using higher-order Hermite interpolation. We advocate 
two-level ROM methods, where a (rational) Krylov method with greedy selection of (a 
small number of) shifts leads to an ROM of moderate size. This ROM can be further 
reduced by applying balanced truncation, or methods in the Loewner framework. 

In mechanical engineering models, the relevant structure can often be preserved 
by not building a frequency-linear ROM, but a model that respects the frequency de- 
pendency by projecting the system matrices on a well-chosen block of the Krylov vec- 
tors. 
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Behzad Nouri, Emad Gad, Michel Nakhla, and Ram Achar 
4 Model order reduction in microelectronics 


Abstract: This chapter deals with the application of model order reduction (MOR) 
in the area of microelectronics. It mainly focuses on the diligent efforts of the MOR 
community in addressing one of the main challenges pertaining to circuit simulation, 
namely, the simulation of high-speed interconnects. A general framework for formu- 
lating the circuit equations that is commonly used in commercial circuit simulators 
is presented. Incorporation of high-speed interconnect structures within the general 
formulation of the circuit equations is described. Current challenges in the MOR of 
interconnect circuits with a large number of ports are presented along with some of 
the recent MOR techniques to handle this kind of circuits. In addition, techniques for 
the reduction of active stable circuits are reviewed with emphasis on guaranteeing the 
stability of the reduced circuits by construction. Several application examples are pre- 
sented to highlight the performance and computational advantages attained by using 
MOR techniques within the circuit simulation environments. 


Keywords: microelectronics, model order reduction, high-speed interconnect, multi- 
port network, stability preservation 
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4.1 Introduction 


The interest in large-scale model order reduction (MOR) in the area of microelectron- 
ics was mainly initiated in the community of electronic design automation in both 
the academic and industrial circles around the early 1990s. This interest was mainly 
spurred by the fast-paced technological development which allowed the very large- 
scale integration (VLSI) of millions of devices on a tiny chip of silicon. With such po- 
tential in the VLSI industry, computer-aided design tools needed to cope, to allow the 
designers to reach closure on their designs, with what is typically described as tight 
market windows. 

The initial thrust for developing MOR in microelectronics was instigated by the 
push in the industry for operating at a higher speed. In fact, semi-conductor fabrica- 
tion technology helped this push through enabling the reduction in device sizes (e. g., 
MOSFET transistors), ultimately leading to reduced processing time. Nonetheless, it 
was the interconnect wires between those devices that represented the dominant fac- 
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Figure 4.1: Electrical interconnects are encountered at all levels of the design hierarchy. 


tor in dictating how high the operating frequencies can go. The reason for this is that 

at high frequency, one can no longer treat interconnects (referred to, henceforth, as 

high-speed interconnects) as simple wires. Rather, their accurate modeling would ne- 
cessitate using the theory of electromagnetic wave propagation, thereby leading to 
large-size networks, whose simulations became oftentimes cumbersome. 

This chapter serves the following two main purposes. 

1. Describing the context in which the need for MOR arose in microelectronics. To 
achieve this purpose, Section 4.2 presents the general circuit formulation in a 
compact mathematical form. This section also highlights the difficulty and the 
high cost associated with fitting the high-speed interconnects in such formula- 
tion. 

2. Presenting an overview of the applications of MOR to address the issues of high- 
speed interconnects in microelectronics at various levels of the design hierarchy. 
As illustrated in Figure 4.1, high-speed interconnects are encountered at all lev- 
els of the design hierarchy, be it on the silicon (on-chip or die), package, board, 
or backplanes level. The application of MOR to high-speed interconnects was 
adapted based on where (which level in the hierarchy) the interconnects are 
used. 


In writing this chapter, the authors focused mainly on the application of MOR to ad- 
dress the challenges of high-speed interconnects in microelectronics. Indeed, from a 
historical perspective, it was those challenges that ushered in the introduction of MOR 
to the area of microelectronics. Nevertheless, the application of MOR in microelectron- 
ics is by no means limited to high-speed interconnects. In fact, it was the reported suc- 
cess in that area that brought it to the attention of the electronic design automation 
community at large and prompted its application to the general area of circuit simula- 
tion. For example, efforts to extend MOR to nonlinear circuits and linear time-varying 
circuits date back to the end of the 1990s (e. g., [55, 101, 94, 72, 99, 27, 117, 46, 45, 74]). 
Another application of MOR in microelectronics was proposed to enable the reduced 
system to capture the original large system along the dimensions spanned by several 
circuit parameters, leading to the parameterized MOR (PMOR), of which the works 
[54, 24] are notable examples. 
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4.2 Formulation of circuits with high-speed 
interconnects in the time domain 


This section presents the modified nodal analysis (MNA) approach that is used by 
virtually all commercial circuit simulators to represent general circuits in the math- 
ematical domain. The presentation of this topic is carried out in two phases. First, the 
MNA formulation for circuits with only lumped elements is considered in Section 4.2.1. 
Next, Section 4.2.2 considers incorporating the high-speed interconnects in the gen- 
eral circuit formulation, dwelling upon the challenges therein, in order to pave the 
way to show the indispensable role that MOR plays in this regard. 


4.2.1 Formulation of circuits with lumped elements 


The presence of nonlinear elements in virtually all circuit designs mandates that the 

natural domain for mathematically describing general circuits is the time domain. 

A widely adopted time-domain formulation is the MNA approach [57, 116, 82]. The 

MNA formulation is derived through: 

1. Writing the Kirchhoff current law at each node. 

2. Expressing the currents in the circuit elements in terms of the node voltages us- 
ing some form of Ohm’s law (i.e., the constitutive relation of the element). The 
node voltages are then considered as the unknowns in the circuit formulation to 
be solved for. 

3. Contriving a special representation for the elements which do not have a simple 
Ohm’s law representation, such as voltage sources, or elements whose constitu- 
tive relation requires integration in the time domain, e. g., inductors. The currents 
in those elements are appended to the set of unknowns to be solved for, along with 
node voltages. Such representation is typically termed “impedance representa- 
tion.” 

4. Representing some nonlinear elements, such as nonlinear capacitors or nonlinear 
inductors, by the charge/flux-oriented formulation in which the charge or flux are 
included in the unknowns [82]. 


By using the above steps, a large system of equations is generated and assembled, tak- 
ing the form of a system of a mixed set of differential and algebraic equations (DAEs) 
that is known as the MNA formulation. Thus, a general circuit with lumped elements, 
such as resistors, inductors, capacitors, etc., is described by the following DAE: 


c0 


g Gx(t) + f(x(t)) = b(t), (4.1) 


where 
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- CG € RM" are real matrices describing the memory and memoryless elements 
in the network, respectively; 

- x(t) € RY is a vector whose components are time-dependent waveforms of (1) 
node voltages (2) currents in elements with impedance representation and (3) 
charges/fluxes in elements with charge- or flux-based representation (e. g., non- 
linear capacitors or inductors); 

- b(t) € R” is a vector of voltage and current waveforms of independent voltage and 
current sources representing the external stimulus of the circuits; 

—  f(x(t)) € R” is a vector whose entries are scalar nonlinear functions RY > R that 
represent the nonlinear elements in the circuit; and 

— Nis the total number of circuit variables in the MNA formulation. 


To solve (4.1) for the circuit variables x(t), various time marching techniques can be 
used. Examples of these techniques are the trapezoidal rule or its high-order variants 
such as the backward-differentiation formulas [116], or the Obreshkov-based methods 
proposed in [47, 121, 31, 32, 73]. 

With typically thousands or millions of circuit elements in modern circuit designs, 
the construction of the MNA formulation can only be performed automatically. The 
approach used in this task is to attach to each circuit element the so-called stamp of 
the element, which describes, so to speak, the “footprint” that the element leaves on 
the mathematical structures (matrices and vectors) of the MNA formulation. The cir- 
cuit is described by a text file typically known as a “netlist.” A netlist can be either 
manually edited by the user or, as in most cases, extracted automatically from circuit 
schematics created by a plethora of commercial software packages such as OrCAD 
PSpice Designer, CADENCE Virtuoso, or MultiSim. The netlist file represents the cir- 
cuit as a sequence of elements, with each line on the netlist file (or a group of lines 
separated by a specially dedicated character) semantically describing one circuit ele- 
ment at a time. Netlists of large circuits can easily grow to millions of lines. Typically, 
the software responsible for automatically constructing the MNA formulation reads 
the netlist one line at a time or, more recently, multiple lines at a time using parallel 
processing. With the syntax used for each line completely identifying the type of cir- 
cuit element (e. g., a capacitor), its value (in Farads), and the circuit nodes to which it 
is connected, the MNA formulation software proceeds, guided by the preprogrammed 
various elements stamps, to add the contribution of each element to the matrices or 
vectors in the MNA formulation in (4.1). 

To provide more insight into this process, we include in the following subsections 
a limited sample of circuit elements and their stamps. 
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4.2.1.1 Resistor’s stamp 


Consider a resistor with resistance R connected between two circuit nodes labeled on 
the netlist n_j and n_jprime (as shown in Figure 4.2(a)), and let j and j’ € N be the 
two (integer) indices associated with the two voltage waveforms of those nodes.! The 
line representing the resistor, such as R = 300 ona netlist may appear as shown by 
the following line: 


R_resistor_label n_j n_jprime R=30 
The stamp of such a resistor is captured using the formulation 


column j column j’ 


G ce G+ row j 1/R —1/R 
rowj! | —1/R 1/R 


4.2.1.2 Inductor’s stamp 


In the same style used for the resistor stamp above, the stamp of an inductor (Fig- 
ure 4.2(b)) with inductance L connected between two nodes labeled on the netlist n_j 


u(t) 


n_jprime 0—— n_jprime n_jprime 


(a) Resistor (b) Inductor (c) Voltage Source 
nj o—o I n_k 
+ 
V, uV, 
n_jprime o——0 n_ kprime 


(d) Voltage-Controlled Voltage Source 


Figure 4.2: A sample of common circuit elements. 


1 In other words x;(¢) = voltage at node n_j, and Xj (t) = voltage at node n_jprime. 
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and n_jprime, with corresponding indices j and j’ € N, may be shown by 


column j column j’ column m+1 
row j +1 
G-—G+ row j’ | -1 | > 
row m+1 +1 -1 
column j column j’ column m+1 
row j 
CeC+ row j’ | | 
row m+1 -L 


The above formulation should be read as follows: The stamp of an inductor re- 
quires appending the inductor’s current waveform to the set of the MNA variables x(t). 
Thus, assuming that there are currently m variables in the formulation, the matrices 
G and C will be appended by an extra row and extra column (m + 1) to account for the 
current in the inductor and its associated constitutive equation. The syntax of the line 
describing an inductor with L = 3 uH typically appears as 


L_inductor_label n_j n_jprime L=3uH 


4.2.1.3 Stamp of an independent voltage source 


Let an independent voltage source (seen in Figure 4.2(c)) have a voltage u(t), and be 
connected between two nodes labeled n_j (positive polarity) and n_jprime (negative 
polarity) with indices j and j’, respectively. The stamp of this voltage source appears 
in both the source vector b(t) and the matrix G, as illustrated by the following formu- 
lation: 


row j 
b(t) — b(t) + row j | | > 
u(t). 


row m+1 
column j column j’ column m+1 


row j +1 
G-G+ row j' | -1 | 
+1 -1 


row m+1 


As can seen from the above formulation, the current waveform in the source enters 
in the set of MNA variables as indicated by the extra row and column in the MNA matrix 
G and source vector b(t). 
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4.2.1.4 Stamp of a voltage-controlled voltage source 


Figure 4.2(d) shows a voltage-controlled voltage source (VCVS). This circuit element 
has four nodes (with indices denoted j, j’, k, k’) and its stamp requires that the cur- 
rent in the controlled source (uV, in Figure 4.2(d)) be included in the vector of MNA 
variables x(t). The following formulation can visualize the stamp of VCVS: 


column j column j) columnk column k’ column m+1 
row j 
row j' 
G-G+ row k +1 
row k' -1 
row m+1 yu H +1 -1 


A typical netlist line representing a VCVS with yp = 0.4 is shown below. 


E_VCVS_label n_j n_jprime n_k n_kprime 0.4 


4.2.2 Incorporating high-speed interconnects in circuit 
formulation 


When the circuit includes (in addition to the above-mentioned lumped elements) a 
subcircuit or subnetwork that is characterized as a high-speed interconnect circuit, 
the mathematical formulation given by (4.1) will have to be amended with a new term 
that significantly alters the nature of the formulation. The reason for that is that those 
networks have their physics grounded in the theory of electromagnetic propagation, 
which is often described by Maxwell’s equations. Under certain assumptions, more 
specifically, a quasi-transverse electromagnetic wave propagation mode [91], the volt- 
ages and currents on those network are properly described as functions of time t and 
a spatial variable z that denotes the spatial distance along the line. 

To further elucidate the above paragraph, Figure 4.3 shows the physical struc- 
ture of a multiconductor interconnect with a ground plane, and Figure 4.4 shows its 
schematic representation as a network of m conductors with 2m ports. 

The schematic representation highlights that voltages and currents are no longer 
localized to a certain node or a certain branch as in the case of lumped elements, but 
are rather distributed along the lines as a function of both time t and distance z. Thus, 
the constitutive relation between the voltage and currents, in this case, is no longer 
an algebraic or simple differential relation, but rather takes the form of a set of partial 
differential equations (PDEs), in which derivatives of the voltages and currents with 
respect to t and z appear. 
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Figure 4.3: A physical representation for 


Ground Plane a high-speed interconnect structure. 
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Figure 4.4: A schematic representation for the high-speed interconnect structure shown in 
Figure 4.3. 


As a consequence of this fact, an attempt to develop a compact stamp, in the same 
spirit as was shown for conventional lumped elements listed above, would be suc- 
cessful only if it is carried out in the Laplace or frequency domain. This is because, 
in the Laplace domain, derivative with respect to t is replaced by the Laplace-domain 
variable s leaving only the derivative with respect to z, therefore, transforming the PDE 
to an ordinary differential equation (ODE) in z, the solution of which can be written 
analytically using the matrix exponential function. 

Unfortunately, such a stamp comes with a complex dependence on the Laplace- 
domain variable (s), which makes deriving the required time-domain stamp a complex 
(if not impossible) task. Indeed, the only possible way to incorporate this stamp in 
the time-domain MNA formulation of (4.1) is to add another term, which involves a 
convolution with x(t). With a convolution term present in the MNA formulation, the 
utilization of time marching to simulate the circuit becomes very inefficient. 

The approaches that are often used to circumvent this difficulty, i.e., to derive a 
time-domain stamp for the interconnect element, can be developed, but usually at the 
cost of compactness. Those approaches, which are collectively known under the term 
“macromodeling,” seek to convert the PDE into a time-domain ODE that can be easily 
stamped into the MNA formulation of (4.1). 
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Macromodeling techniques typically approach the problem by discretizing the 
PDE along the spatial dimension z, approximating the z derivative using an appropri- 
ate approximation operator. Examples of these techniques are matrix rational approx- 
imation (MRA) [28, 29] of the exponential function and delay-based approaches such 
as method of characteristics (MoC) [53] and the DEPACT algorithm [83]. Macromodel- 
ing, used in that sense, solves the issue of stamping the high-speed interconnect in the 
MNA formulation, but at the cost of compactness as it produces a large network of ex- 
tra circuit elements. The goal of using MOR in high-speed interconnects is to address 
the complexity of simulating the circuit with large macromodels. 

The next subsection presents the macromodeling based on lumped segmentation, 
while the rest of the chapter will review the application of various MOR approaches 
used to reduce the complexity of the models. 


4.2.3 Time-domain macromodeling based on discretization 


Discretization techniques represent a very straightforward approach to overcome the 
above difficulties and incorporate high-speed interconnects in circuit simulators. Fol- 
lowing the central idea of these techniques, one first needs to introduce the param- 
eters of the line, which are the resistance (R), inductance (L), conductance (G), and 
capacitance (C) per unit length. In the case of an interconnect with m conductors, 
those parameters are m x m matrices. 

Discretization proceeds by dividing the line into segments of length Az, chosen 
to be a small fraction of the shortest wavelength in the driving signal.’ If the length 
of each of these segments is electrically small (i.e., compared to the shortest wave- 
length), then each segment can be replaced by the model shown in Figure 4.5. 

It is of practical interest to know how many of these segments are required to rea- 
sonably approximate the interconnect. For illustration consider a lossless line, i.e., 
R = Oand G = O, with only LC elements which can be viewed as a low-pass filter. For a 
reasonable approximation, this filter must pass at least some multiples of the highest 


RAz LAz 
CAz GA: 
o | 
E Figure 4.5: Modeling a segment of a 
4 | single-conductor transmission line 
Az using lumped circuit elements. 


2 The shortest wavelength is obtained by dividing the speed of light in the medium of the interconnect 
by the highest frequency in the propagating signal. 
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frequency fmax Of the propagating signal (say, 10 times, fọ > 10f,,9,). To relate these 
parameters, we make use of the 3 dB passband frequency of the LC filter given by [63] 


1 1 


= paca ae Ta 


fo 
where d is the length of the line and r = VLC represents the delay per unit length 
(p. u.1.). Typically, we set fmax = 0.35/t,, where t, is the rise time of the signal. Using 
(4.2), we can express the relation fọ > 10fmax in terms of the delay of the line and the 
rise time as 1/(atd) > 10 x 0.35/t, or t, = 3.5(atd) =~ 10rd. In other words, the delay 
allowed per segment is approximately t,/10. Hence the total number of segments (P) 
needed to represent the total delay of td is given by 


P =Td/(t,/10) = 10Td/t,. (4.3) 


As an example, consider a digital signal with rise time t, = 0.2ns propagating on 
a lossless wire of length 10 cm witha p. u. 1. delay of 70.7 ps, which can be represented 
by a distributed model with p. u.1. parameters of L = 5nH/cm and C = 1pF/cm. If the 
same transmission line were to be represented by lumped segments, one needs P = 35 
segments. 

One of the major drawbacks of the above approach is that it requires a large num- 
ber of sections, especially for circuits with many multiple conductors, high operating 
speeds, and sharper rise times. This leads to large circuit sizes and the simulation be- 
comes CPU-inefficient. 


4.3 Model order reduction: application perspective 


A methodological approach to describe the application of MOR in microelectronics 
can begin by considering the MNA formulation for general time-invariant large-scale 
continuous circuits as presented in (4.1). In the absence of nonlinear elements, a gen- 
eral linear circuit with several input and output terminals can be described by ne- 
glecting the nonlinear term in (4.1). Additional mathematical terms are also included 
to delineate their input-to-output behavior. In particular, to describe a network with 
only linear elements, having ni, inputs and nout outputs, the MNA formulation takes 
on the following form: 


CÉ x(t) + Gx(t) = Bulo, (4.4a) 
y(t) = L'x(t), (4.4b) 


where x(t) € RY, C,G € RYN, B e RN*"», and L e RN*Mout, 
For engineering applications such as microelectronics, microelectromechanical 
systems, and electromagnetism, the size of the system in (4.4) can be very large, in 
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the range of millions. To reduce the computational cost associated with such large 
networks, MOR [6, 14, 113, 105, 13] has proven to be an effective tool to develop simpler 
models that capture the essential features of the given large system and accurately 
approximate their input-to-output behavior. The resulting model replaces the complex 
original system to ensure fast and reliable simulations. 


4.3.1 Explicit moment-matching 


Given a linear time-invariant (LTI) circuit in MNA form (4.4), its input-to-output rela- 
tionship under the assumption of zero initial state can be described in the Laplace 
domain as Y(s) = H(s) U(s), where 


H(s) = L'(G +sC) 1B (4.5) 


is acomplex-valued transfer function. The Laplace variable “s” is in the form of a com- 
plex frequency. The importance of transfer functions in characterizing LTI systems 
stems from the fact that the unit-impulse response of the system in the time domain 
can be recovered by the inverse Laplace transform of H(s). 

A natural idea for MOR is to construct a reduced-order model such that the Taylor 
series expansion of its transfer function H(s), with respect to s, matches a number of 
the leading terms in the Taylor series expansion of the original H(s) (4.5). Assume G 
is invertible; then DC (s = O rad/s) can be a prompt choice for the expansion point. 
Taylor series expansion of H(s) around s = 0 can be obtained by expanding X(s) as 


H(s) = L'X(s) = L' Y M; s', (4.6) 
i=0 


where the coefficient of s' in the expansion is called the i-th moment at s = 0. 


4.3.2 Moment computation in MNA formulation 


Applying Laplace transform to (4.4a), assuming the unit-impulse excitations at the 
inputs, and expanding its X(s) using Taylor series at s = 0, we obtain 


(G+sC)(M) + M,s + M? + --- + Mjs' + ---) =B. (4.7) 
Moments of the MNA variables X(s) are derived in a recursive form as 
M;i; — AM; (4.8) 


where 


Mo =R, R=G'B, A=-G'C. (4.9) 
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Computation of moments only need one LU decomposition of G. The moment compu- 
tation formula can be equivalently written as 


M,=A'R, for i=0,1,2.... (4.10) 


4.3.3 Asymptotic waveform evaluation 


Asymptotic waveform evaluation (AWE) [98, 96, 21] was the first MOR method that was 
based on the moment-matching idea. Following the steps of AWE, first, 2m leading 
moments of the circuit are obtained through a fast recursive moment computation. 
These 2m explicit moments (4.10) are used to find m poles and residues of the m-th- 
order macromodel via the Padé approximation. 

The explicit moment-matching approaches, namely, AWE, suffer from numerical 
limitations and become ineffective due to the inherent ill-conditioning nature of ex- 
plicit moment generation. Thus, these methods can be used for small-order approxi- 
mations which require only a few moments to be matched. 


4.3.4 Projection for order reduction 


An elegant solution to overcome the innate ill-conditioning of the methods based on 
explicit moment generation is to use projection-based MOR methods, which are based 
on implicit moment-matching [34, 38, 40, 111, 88, 20, 50, 25]. The enabling idea is to 
first reduce the number of MNA variables by projecting x(t) to smaller subspace, as 
shown in (4.11). For this, an orthogonal projection matrix V € RY with m « N is 
used as 


x(t) = Vx(t). (4.11) 


RNxm 


Then, a left-projection matrix U € is used, in general, to reduce the size of the 


resulting circuit equation as 
vicv Sa) + U'GVX(t) = U'Bu(t). (4.12) 


Given the projection matrices U and V (€ R‘*”), the reduced-order model for the 
MNA formulation in (4.4), obtained through a Petrov—Galerkin projection scheme [6], 
can be formalized as 


ET + Gx(t) = Bu(t), (4.13a) 
y(t) = L'X(6), (4.13b) 


where 
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€=uU'cv c R™", G =U'GV e R™”, (4.13c) 
Ê = UB e R™", Lt =L'V ¢ R”, (4.13d) 


The model reduction methods differ in the choice of their projection matrices U and V. 
For the techniques based on the idea of split congruence transformations [65], such 
as passive reduced-order interconnect macromodeling algorithm (PRIMA) [88], both 
projection matrices span the same subspace and thus a single projection matrix is 
used, U = V. 

Computation of the orthogonal basis V is an essential step in the split congruence 
transformation-based reduction process and can be the main differentiation factor be- 
tween the reduction methods. 


4.3.5 Krylov subspace techniques 


The Krylov subspace is defined as a subspace of the space spanned with the columns 
of block moments of circuit response, as given in (4.14). Let V € R™*" be the sought 
orthogonal basis matrix spanning the Krylov subspace as 


colspan(V) = r(A, R, q) = colspan{R, AR,.. ATOR}, (4.14) 


such that Vt V = Imxm (orthogonal), where m = q x nin. Using the (block) Krylov basis 
V as a projection matrix is a common approach in MOR [38, 20, 8, 50]. The Lanczos 
algorithm [69] and Arnoldi process [7] are two numerically robust methods to generate 
the Krylov basis matrix V. 

The Padé via Lanczos (PVL) method was the first projection-based method [34] 
to implicitly match the reduced model and the original system to a certain order of 
moments [51]. The matrix PVL (MPVL) algorithm is an extension of PVL to general 
multiple-input multiple-output systems [35, 3]. To deal with circuits with symmetric 
matrices, the SyPVL algorithm [42] or its multiport counterpart (SyMPVL) [43] were 
developed. 


4.3.6 Arnoldi algorithm 


The Arnoldi process using the modified Gram-Schmidt orthogonalization recur- 
sively produces a set of orthonormal vectors as the basis for a given Krylov subspace 


Kr(A,R, q) [7]. The algorithm generates an orthogonal projection matrix V € R*” 
and a block upper Hessenberg matrix H € R”*” which satisfy 
VIAV = H. (4.15) 


For the practical implementations of the Arnoldi algorithm for single-input single- 
output systems and the block-Arnoldi algorithm for the multiple-input multiple- 
output cases, one can refer to [102, 20, 1]. 
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4.4 Formulation of RC circuits representing on-chip 
interconnects 


As stated in Section 4.1 and shown in Figure 4.1, the presence of high-speed intercon- 
nects is ubiquitous at all levels of the electronics design hierarchy, be it on the sili- 
con (on-chip or die), package, board, or backplanes level. It is often the case that the 
on-chip interconnects are characterized by increased resistance and negligible induc- 
tance. In this situation, a mesh of RC elements can accurately model the on-chip inter- 
connect. Based on a general MNA circuit formulation (4.4), the impedance parameter 
realization for RC network presenting an on-chip interconnect is obtained with L = B 
as 


Gx(t) = -c20 + Bu(t), z(t) = B'x(t), (4.16) 


where G and C are symmetric and respectively contain the stamps of parasitic resistors 
and capacitors and u(t) contains the current source excitations at input terminals. The 
capacitors being real and positive, matrix C is positive semi-definite. For this realiza- 
tion, G has positive diagonal entries that are greater than or equal to the sum of the 
absolute value of the off-diagonal elements in its row, and are so-called irreducibly 
diagonally dominant [82]. This means that none of the eigenvalues of either matrix is 
negative. 


4.4.1 Reduced RC macromodel for on-chip interconnects 


The central idea to obtain an efficient and accurate macromodel for the RC intercon- 
nect in (4.16) is to exploit the symmetry and positive definiteness properties of its MNA 
matrices. Assuming G is invertible, and hence the circuit has a DC solution, it is 


G=G'>0 and C=C's0. (4.17) 


A circuit with matrices possessing the properties in (4.17) is referred to as a sym- 
metric system. For symmetric positive definite G shown in (4.17), the Cholesky factor- 
ization exists as [49] 


G = GiGi, (4.18) 


where G; is lower triangular with positive diagonal elements. By substituting (4.18) in 
(4.16) and defining J 2 Gi", we get 


x(t) = sep ZO +JBu(t), z(t) = (JB)'x(t). (4.19) 
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Using the projection matrix V obtained from running the block Arnoldi algorithm with 
matrix R = JB and the symmetric negative semi-definite matrix A = -Jjcy', considering 
(4.15), the reduced macromodel for (4.19) is constructed as 

X(t) = Xo + Bu(t), z(t) = B'x(t), (4.20) 
where A = -VCJ'V € R™ is symmetric and block tridiagonal and Ê = V'JB. The 
macromodel in (4.20) is an m-th-order system matching the first 2g moments of the 
original system. 


Proposition 4.1. For any symmetric system, using a projection matrix formed by the 
Krylov bases as V = Kr(-JCJt, JB, m), the first 2q (block) moments of the original and 
reduced-order system of order m = q x nj, match. 


The proof is possible by induction and is straightforward by following the similar 
steps presented in [103]. 

An attempt to generalizing this approach to the cases of RL and LC circuits has 
been presented in [44]. 


4.5 Model order reduction of RLC on-chip 
interconnects 


As a long interconnect is imposed with faster on-chip rise times, the impact of its in- 
ductive property becomes noticeable. The wide wires which are frequently encoun- 
tered in clock distribution networks and upper metal layers can be considered as typ- 
ical examples. These wires, having a low resistance, exhibit inductive effects with a 
dominant impact on signal propagation. In these cases, inductance cannot be ne- 
glected anymore and needs to be included in the models for realistic simulations of 
VLSI designs. To handle the resulting extremely large RLC circuit models for on-chip 
interconnects, several algorithms are available in the literature [110, 67, 37]. While 
these methods can produce an accurate reduction for RLC networks, they cannot guar- 
antee the passivity. However, preserving passivity in the reduction of general RLC net- 
works is a practical necessity. Passivity implies that a network cannot generate more 
energy than it absorbs from its sources. It is important because the cascade connec- 
tions of (strictly) passive circuits will be (asymptotically) stable [19]. However, inter- 
connections of stable but nonpassive macromodels may not necessarily be stable. For 
a detailed account of passivity and the importance of passivity preservation, [12, Chap- 
ter 5] can be referred to. 
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4.5.1 Passive reduced-order interconnect macromodeling 
algorithm 


To establish the idea, let us consider the admittance parameter realization [20] for a 
general multiport RLC network in the time domain described using MNA circuit equa- 
tions in the form presented in (4.4) with L = B, where u(t) contains the voltage source 
excitations at input terminals and outputs are the currents entering the same termi- 
nal. Logical partitioning of the unknown vector x can be given as X = [Vyodes> Ipranches!s 
where Vj oges İS voltages at the nodes and ibranches Contains the currents in the branches 
of inductors and voltage sources. Correspondingly, G, C, and B in (4.4) can be parti- 
tioned as 


G G C 0 B 
G= 11 2 i C= | 11 | , B= | | i 4.21 
E 0 0 Cy» B» (eeN 


where G4; is symmetric positive definite provided each internal node has a DC path (to 
the ground), G, is a block containing zeros and ones, C,, is symmetric and positive 
semi-definite, and C,, is symmetric negative semi-definite. The passive reduced-order 
interconnect macromodeling algorithm (PRIMA) [20, 88] is a Krylov subspace-based 
projection method using the Arnoldi process. By taking advantage of the particular 
block structure of linear RLC circuits as given in (4.21), PRIMA creates passive reduced- 
order models. The passivity preservation requires a simple modification in the circuit 
formulation as negating the rows in conductance matrix G, susceptance matrix C, and 
input matrix B corresponding to the current variables as [20, 110] 


G, G c 0 B 
G=| 2 ae c-| 1l |: B=] ap 4.22 
a 0 0 -Cy -B, ee 


Using the modified matrices in (4.22) and the projection matrix from the Arnoldi pro- 
cess, the reduced model in the form presented in (4.13) can be computed. The following 
theoretical results can be stated for the resulting reduced macromodel. 


Corollary 4.1 (Preservation of moments). Given an mxm Krylov basis projection matrix 
as V = Kr(-G"'C, GIR), the reduced PRIMA macromodel in (4.13) preserves the first 
q = [m/n]? block moments of the original system. 


For the proof, [20, 88] can be referred to. 


4.5.1.1 Passivity of the reduced model 


A linear network is passive if its admittance or impedance transfer function matrix 
H(s) is positive real by satisfying the following necessary and sufficient conditions 


3 |x] Floor operator rounds down x to the largest integer number less than or equal to x. 
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[5, 84, 18]: 
H(s) is defined and analytic in Re(s) > 0, (a) 
H*(s) = H(s*), (b) (4.23) 
@(s) = (H(s) +H*(s))>0 Vs € C:Re(s) > 0, (c) 


where superscript « is the complex conjugate operator. For the reduced models the real 
reduced matrices C, G, Ê, and L are real, the passivity can be equivalently investigated 
by checking [68, 88] 


@(s) = (H(s) + H'(s*))>0 Vs € C: Re(s) > 0. (4.24) 


For the macromodel such as impedance and admittance realizations where L = B, we 
have the following. 


Corollary 4.2 (Preservation of passivity). Given a continuous-time linear system with 
the real-valued matrices (G + Gt) > O and Ct = C > O and any full rank reduction 
projection matrix V € IRN*", the reduced PRIMA macromodel is passive. 


The proof is possible by showing the satisfaction of the passivity condition in 
(4.24) following the steps presented in [20, 88]. 

From Corollary 4.1, it is seen, after selecting the required number of block mo- 
ments g to achieve a desired predefined accuracy, that the order of the reduced system 
m proportionally increases with the increase in the number of ports, i. e., m = q X Nip. 


4.5.2 Example: reduction using PRIMA 


In this example, we consider an RLC mesh shown in Figure 4.6. It is connected to the 
rest of the circuit through its 24 ports. The order of the subcircuit (excluding termina- 
tions) is N = 5,800. The original subcircuit is reduced using the PRIMA algorithm to 
form a passive reduced macromodel of order m = 290. 
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Figure 4.6: A network including a 24-port RLC mesh as its subcircuit (Section 4.5.2). 
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Figure 4.7: Transient responses at the terminal at the left of the horizontal trace#1 (top) and trace#10 
(bottom) (Section 4.5.2). 


The reduced model is plugged in the MNA equations of the rest of the circuit, which has 
three input voltage sources connected to the near-ends terminals 1, 6, and 12. The test 
excitations are set to trapezoidal pulses with rise/fall times of 0.1ns, a delay of 1ns, 
and a pulse width of 5ns. The simulation results obtained from the original circuit 
of Figure 4.6 and from the network using the reduced macromodel are compared in 
Figure 4.7, which shows excellent agreement. 


4.5.3 Structure-preserving model order reduction of RLC 
interconnects 


The prominence of the PRIMA method presented in Section 4.5.1 is mainly due to 
its passivity preservation. As previously described in Section 4.5.1, to create passive 
reduced-order models, PRIMA relies on the special block structure of linear RLC cir- 
cuits (4.22). Later, in [40, 39, 41], the structure-preserving reduced-order interconnect 
macromodeling (SPRIM) method was developed. Besides passivity, SPRIM can pre- 
serve other characteristics inherent to RLC circuits, such as the block structure of the 
circuit matrices and the reciprocity. 

In the SPRIM approach, first, a projection matrix V € is obtained by run- 
ning the Arnoldi process. According to the block structure of the system matrices, the 
projection matrix is partitioned to 


RNxm 


V= H . (4.25) 


Next, the blocks V, and V, are rearranged to form a new projection matrix as 


A V (0) 
=| RO, 2 
i le (4.26) 
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Using V € RY”? (4.26) and based on the concurrence transformation, the reduced 
model is obtained. 

It can be shown that the SPRIM constructs the passive reduced-order models, 
which preserve twice as many moments as the corresponding PRIMA models obtained 
with (almost) the same computational cost. Hence, SPRIM offers comparable accu- 
racy with the Padé-type approximation in the sense that both match twice as many 
moments. For the real frequency expansion point Sọ, SPRIM is more accurate than 
PRIMA. The SPRIM model, written in the form of first-order DAEs, would be twice as 
large as the corresponding PRIMA model. However, the SPRIM model can always be 
represented in the second-order form with the same size as the PRIMA model. 


4.6 Model order reduction of RLC interconnect 
structures with many ports 


For designers to accurately assess on-chip layout-dependent parasitics before fabrica- 
tion, extremely large RLC representations for on-chip interconnects are automatically 
extracted from layout and are included as subnetworks in the netlist for circuit simu- 
lation. Often these subnetworks have many interfaces with other parts of the on-chip 
design. Some challenges arise in the MOR of such RLC networks with a large num- 
ber (e. g., thousands) of input/output terminals. The direct application of the conven- 
tional implicit moment-matching MOR techniques such as PRIMA and SPRIM on a 
multiport network often leads to inefficient transient simulations due to the large and 
dense reduced models. As shown in Section 4.5.1, to achieve the desired accuracy, for 
every increase in the number of ports, the order of the reduced system increases pro- 
portionally to the number of block moments. 

Several attempts have been made to confront this problem via port-compression. 
Early studies in [33, 36] reveal that there may exist a large degree of correlation be- 
tween various input and output terminals. Incorporating this correlation information 
in the matrix transfer function at the I/O ports of the reduced model during the reduc- 
tion process became the common theme in the existing terminal-reduction methods. 
However, the major difficulty in port-compression algorithms such as SVDMOR [33], 
ESVDMOR [79], RecMOR [36], and several others [15, 78, 76, 80, 77] is that the correla- 
tion relationship is frequency-dependent and in many cases also input-dependent. In 
general, practical networks with many ports rarely exhibit a high degree of correlation 
[118]. As a consequence, such a reduction can lead to accuracy loss. 

Also, due to the importance of on-chip RC interconnect, various efforts have been 
reported in the literature tackling this issue for the case of RC networks, such as [66, 
119, 60, 89]. 

Recently, for the general case of on-chip interconnect RLC circuits with a large 
number of ports, an efficient MOR has been presented in [86]. This method exploiting 
the superposition paradigm [106, 11] proposes a reduction strategy based on a flexible 
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clustering of inputs. Thereby, the problem of reducing networks with many ports is 
simplified by clustering inputs into small groups, and reducing each subsystem indi- 
vidually. Next, the reduced subsystems are concatenated to constitute a global macro- 
model resulting in an accurate and sparse block-diagonal reduced model (see the right 
graph in Figure 4.10), which is stable by construction. 

Since subsystems are treated independently, passivity is not always guaranteed. 
However, the flexible clustering scheme of the method, along with the information 
from the geometry of the design, is used to improve the passivity of the resulting 
model. The flexibility of the technique allows passivity preservation to be considered 
as the primary criterion when grouping the lines into the clusters. Utilizing well- 
established passivity enforcement techniques as presented in [12, Chapter 5] can also 
be considered as an alternative strategy. To this end, based on thea posteriori passivity 
check, a postprocessing procedure can be applied to enforce the model passivity. 


4.6.1 Example: reduction of multiport network 


The multiconductor interconnect circuit shown in Figure 4.8 has 64 terminals through 
which it is connected to the rest of the design. 


Length =10 cm 


soo i Line 1 


509 ; Line 16 


l : Cp=1pF 
502} Line 17 : I 


50Q i Line 32 


Figure 4.8: 32-conductor coupled interconnect network with terminations (Section 4.6.1). 


Applying the multiport-MOR method in [86] to the network in Figure 4.8, a multiport 
reduced-order model is obtained. Figure 4.9 demonstrates the accuracy of the result- 
ing reduced model by sample comparisons of time-domain responses, depicting an 
excellent agreement of the responses. 
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Figure 4.9: Transient responses at victim line near-end of line#2 (top) and far-end of line#31 (bottom) 
(Section 4.6.1). 
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Figure 4.10: (left) Sparsity pattern of reduced MNA equations using conventional PRIMA (dense). 
(right) Sparsity pattern of reduced MNA equations using the method in [86] (Section 4.6.1). 


Figure 4.10 illustrates the block diagonal structure of the resulting reduced model com- 
pared to using the conventional PRIMA algorithm, which admits a significant sparsity 
advantage. 

Table 4.1 compares the CPU time expense for the transient simulation of the mul- 
tiport circuit in Figure 4.9 using different approaches. As seen, while applying PRIMA 
leads to a macromodel that is prohibitively expensive even compared to the original 


Table 4.1: CPU cost comparison between the original system, PRIMA, and the method in [86]. 


Original PRIMA Method in [86] 


Size 29.195 2.560 2.560 
Total CPU time (s) 645.9 1730 111.7 
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circuit, the multiport reduction algorithm in [86] achieves a speedup factor of 15.5 com- 
pared to PRIMA. 


4.7 Model order reduction of active circuits 


Active circuits are often used (inserted) along the on-chip interconnects for example 
to minimize the propagation delay of the signals transmitted through those intercon- 
nect lines [61, 9, 115, 10, 2, 112, 26, 59, 4]. An illustration of the idea is presented in 
Figure 4.11. 


Figure 4.11: Amplifiers inserted in an RLC line to minimize the propagation delay by dividing the 
interconnect line into shorter sections [61]. 


The active circuits generally include dependent voltage and/or dependent current 
sources, e. g., in small-signal device models, amplifier circuits, etc. Considering the 
stamps of dependent sources as presented in Section 4.2.1.4, it is straightforward to 
see that in the presence of these sources, (G + G') is not always positive semi-definite. 
For such circuits, the congruence transformation does not guarantee the stability of 
the reduced models. Therefore, using conventional MOR techniques such as PRIMA 
(Section 4.5.1) and SPRIM (Section 4.5.3) for the reduction of these circuits will not 
guarantee the stability of the resulting model. However, it is desirable and often cru- 
cial that resulting reduced-order models inherit the stability of the original circuit. 
Unstable models can lead to inaccurate or totally unfeasible time-domain simulation. 

There are a few approaches for stability preservation of the resulting reduced mod- 
els, such as [107, 17, 75, 64], which are usually based on postprocessing. For example, 
in [64, 62, 90] a method is proposed to eliminate the unstable poles of the reduced 
system by using implicitly and explicitly restarted Arnoldi and Lanczos algorithms. 
However, the common concept in these methods is to sacrifice accuracy of the reduced 
model in order to guarantee stability, which may destroy the integrity of the moment- 
matching algorithm leading to an inaccurate reduced model [107]. In addition, nu- 
merical algorithms for restoring stability [17] are not guaranteed to converge, and in 
general, they have a relatively high computational cost associated with them. Classi- 
cal truncated balanced realization (TBR) [81, 6] is another method to preserve stability. 
However, it is computationally expensive, which makes it not suitable for very large 
circuits. In addition, the existence of a solution for general circuit formulation is not 
guaranteed. 
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In [87] a projection framework is presented for constructing stable reduced 
macromodels for stable active linear circuits. To this end, the right-projection ma- 
trix V € R™™ is formed through implicitly matching the first m moments of the 
original circuit equations as described in Sections 4.3.5 and 4.3.6. Next, a full rank 
left-projection matrix U € R*"" is constructed through implicitly satisfying a stability 
condition in the form of a generalized Lyapunov inequality [6, 17]. 

Given the generalized eigenvalue and eigenvector matrices of the matrix pencil 
(c',-G') as D € C”™ and I € C", respectively, which are obtained by generalized 
eigenvalue decomposition as 


c'r = -G'rD, (4.27) 


a full column rank left-projection matrix can be formed as 


U=IF,, 5 (Fe ET; IT, =P) |-er (4.28) 
where I, contains the real eigenvectors corresponding to the m, real eigenvalues in D 
and I, and? contain the complex eigenvectors corresponding to the m, complex and 
conjugate eigenvalues of D. The computational steps to generate stability-preserving 
left-projection matrix are illustrated in Algorithm 4.1. 

The resulting left-projection matrix U plays the role of guaranteeing stability of the 
reduced-order model by ensuring that the Lyapunov stability constraint is satisfied by 
the resulting reduced model. Thereby, the proposed algorithm guarantees the stability 
of the reduced model by construction without numerical optimization or postprocess- 
ing. 


4.7.1 Example: reduction of active circuit 


We consider an amplifier circuit shown in Figure 4.12 as an example of large active cir- 
cuit. The interconnect structure consists of four coupled lines of length L = 10 cm. The 
high-frequency equivalent-circuit model for amplifier blocks is shown in Figure 4.13. 

The transmission structure whose geometry is shown in Figure 4.12, is modeled us- 
ing lumped RLGC segmentation with p. u.1. parameters obtained from HSPICE. Apply- 
ing the MOR method in Section 4.7, a stable reduced-order model for the original (sta- 
ble) network in Figure 4.12 is obtained. The accuracy of the resulting reduced model is 
demonstrated in Figure 4.14, by comparing the transient voltage responses of the re- 
duced model with the simulation results of the original (unreduced) model. The input 
signal is trapezoidal-pulse with the delay time t4 = 1ns, pulse width ¢,,, = 5ns, and 
the rise and fall times t, = 0.25 ns and ty = 0.25 ns, respectively. 

The resulting reduced model provided an accurate time-domain response, where- 
as the time-domain simulation of the Arnoldi-based model failed to converge because 
of the unstable poles. 
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Algorithm 4.1: The proposed method for stable model order reduction of active 
circuits. 
Input: Original G, C, B, L, Reduction-order m. 
Output: Stable reduced-order model: G, C, Ê, L. 
V — Amoldi(-G'C, G"'B, m); // right-projection matrix 
rDc eigs(C', -Gt, m); // Sparse-generalized eigenproblem solver 
D +- diag(D); 
if D(m) ¢ R then 
if D(m) + D(m-1)* then 
| Tm) =[]; 
end 


w N = 


end 


o oo NAUA AA 


i-l; 
10 while i < m do 
11 Yi -T T(;,i); // y; is the i-th column vector in T 


12 if D(i)€ R then 


add to 
13 U — Yi; 


14 i—i+1; 
15 else 


16 Ue Rely;), Zm(y;); 


17 ic i+2; 


18 end 
19 end 
2 C—UcV, G-UGV, Be UB, L-LV; 


Table 4.2: Comparison of the original and reduced models (Section 4.7.1). 


Dimension Stability 


Original circuit 12,024 Yes 
Arnoldi-based reduced model 46 No 
Proposed reduced model 46 Yes 


The sizes and stability properties of the original and reduced models are compared in 
Table 4.2. 

The CPU-time for frequency-domain simulation of the original is compared with 
the proposed reduced model in Table 4.3. 
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Figure 4.12: A stable active design consisting of four coupled interconnects and amplifier blocks 
(top), Cross-section of transmission line structure (bottom) — (Section 4.7.1). 


Table 4.3: Comparison of the CPU time for frequency simulation using original and reduced models 
(Section 4.7.1). 


Original Reduced 


Order 12,024 46 
Model generation time (s) - 0.393 
Simulation time (s) 33.306 0.195 
Speedup factor =57 
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Figure 4.13: High-frequency equivalent-circuit model for MOS cascade amplifier (the biasing network 
is not presented) (Section 4.7.1). 
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Figure 4.14: Comparison of the transient responses at Vout at the far-end of line 3 (top), and at Vout, 
at the far-end of line 4 (bottom) (Section 4.7.1). 


4.8 Conclusions 


This chapter described the application of MOR for efficient analysis of microelectronic 
structures in circuit simulation environments. A general framework for formulating 
the circuit equations based on the MNA approach that is commonly used in commer- 
cial circuit simulators was presented. The alternative means to incorporate high-speed 
interconnect structures within the general formulation of the circuit equations were 
introduced. This leads, in general, to circuit models with a large number of lumped 
components. MOR techniques for RC and RLC interconnect circuits with emphasis on 
stability and passivity preservation were reviewed. Current challenges in the MOR of 
interconnect circuits with a large number of ports were presented along with some of 
the recent MOR techniques to handle this kind of circuits. In addition, existing tech- 
niques for the reduction of active stable circuits were reviewed with emphasis on guar- 
anteeing the stability of the reduced circuits by construction. 

It should be noted here that the presentations of MOR application in high-speed 
interconnects have been restricted to those techniques that are based on projecting 
the system into its Krylov subspace. Other projection-based methods have also been 
proposed to handle the high-speed interconnect. Worthy of note among those meth- 
ods are the truncated balanced realization algorithms [48, 71, 93, 92, 70, 95]. Another 
class of projection-based methods known as proper orthogonal decomposition (POD) 
or principal component analysis were proposed for both linear and nonlinear systems 
[16, 58]. However, they have not been widely applied to microelectronics. 

In addition to the above projection methods, there are also nonprojection meth- 
ods, of which the explicit moment-matching of Section 4.3.1 is a known example. There 
are other well-known approaches based on the Hankel norm of the system [48, 104]. 
Another group of nonprojection approaches relies on fitting the transfer function of 
the system [23] in the frequency domain, of which the method based on vector fit- 
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ting [56, 52, 85] is widely adopted in high-speed interconnects. An alternative class of 
nonprojection methods is constituted using the basic idea behind a method termed 
“selective node elimination” [30]. Several methods related to this approach have been 
developed in the literature (e. g., [97, 109, 114, 120, 100]), and the time constant equili- 
bration reduction [108] for the reduction of RC networks, which was extended to RLC 
circuits in [22]. 
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systems 


Abstract: This chapter has two main objectives: first, to propose a computer-aided con- 
sistent and accurate description of the behavior of electromagnetic devices at various 
speeds or frequencies and, second, to describe procedures to generate compact elec- 
trical circuits for them, with an approximatively equivalent behavior. The extracted 
models should have a finite complexity as low as possible, while yielding an accept- 
able accuracy, as well as preserve essential characteristics, such as passivity. A suc- 
cessful complexity reduction can be obtained if a priori and on-the-fly reduction strate- 
gies are applied before and during the model discretization, followed by a posteriori 
complexity reduction. 


Keywords: electromagnetic devices, electromagnetic systems, conceptual models, 
mathematical models, numerical models, a priori reduction, on-the-fly reduction, 
a posteriori reduction 
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5.1 Introduction 


An electromagnetic (EM) device is a system in which the EM field plays an essential 
role. In order to extract a model of finite complexity, the first objective is to reduce the 
complexity of the physical and mathematical models, which are initially of infinite di- 
mension. The reduction from infinity to a finite order can be achieved by discretization 
with a numerical method. However, due to the structural and geometrical complexity 
of devices encountered now in real-life, the discrete models thus obtained have an ex- 
tremely high order of complexity, reaching even orders of magnitude above millions, 
requiring additional complexity reduction. That is why it is important to reduce the 
model complexity not only after numerical discretization, but also prior to it, during 
the modeling stage. 

The modeling procedure consists of seven successive steps, each dedicated to 
building a particular form of the model: (1) Conceptual modeling (conceptual model = 
geometrical model + physical model) establishes consistent geometrical models and 
physical models of the device including simplifying hypotheses and justified approxi- 
mations of geometrical and physical nature. (2) Mathematical modeling formulates the 
mathematical equations that describe the operation of the device, presenting the con- 
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ceptual model in a mathematical language as a properly formulated problem. (3) Ap- 
proximate analytical modeling (AAM) determines the approximate analytical relation- 
ship between the input and output physical quantities by solving an approximate, 
simplified version of the model equations. (4) Numerical modeling aims to build an 
algorithm to obtain the solutions of the mathematical model equations. (5) Computa- 
tional modeling (simulation software) aims to create and test a computer program to 
implement the numerical algorithm conceived in the previous step. (6) Model order 
reduction (MOR) finds the simplest input/output system which approximates a poste- 
riori, i. e., after the discretization, the behavior of the modeled device, while maintain- 
ing the original behavior with an acceptable accuracy. (7) Verification and validation 
is the final step of the procedure, consisting of the verification of the solution obtained 
by simulation and the validation of the extracted model, by comparison with experi- 
ments. Without this final check the entire procedure is completely useless. 

The discretization methods used in the fourth step and their main characteristics 
are given in Table 5.1. Since the most used one is the finite element method (FEM), the 
final section of this chapter is devoted solely to it. 


Table 5.1: Characteristics of the fundamental numerical methods for EM field analysis. 


Method Discretization mesh Form of discretized equations 

FEM Unstructured, formed with triangles, Weak form, 

Finite element quadrilaterals, tetrahedrons, variational equations 
hexahedrons, etc. 

FDM, FIT Grid — mesh with regular topology, Differential (FDM) 

Finite differences, Pair of staggered dual grids in the case of or global equations (FIT) 

Finite integrals hyperbolic equations 

BEM Unstructured two-dimensional mesh, on Integral equations 

Boundary elements the domain boundary 


The complexity of the EM model in steps 1 and 2, which has distributed parameters and 
is described by partial differential equations (PDEs), can be reduced not only a posteri- 
ori, after discretization, but also on-the-fly, during the discretization, or even a priori. 
All these complexity reduction steps contribute equally to obtain, eventually, a model 
appropriate for the designer’s needs. From this perspective, it becomes apparent that 
the efficient reduction of the model complexity cannot be limited to a reduction ap- 
plied to the discrete model with classical MOR methods, or by matrix condensation, 
and that the best final results require reductions before and during discretization. 
This chapter is structured according to the modeling procedure above. 
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5.2 Fundamental quantities and equations 


In the macroscopic electromagnetism, the spatial distribution of the EM field is de- 
scribed at each moment in time by a first set of four local primitive physical quantities 
[43], which are the alphabet of this theory: (1) electric field strength E= fi (7, t) (V/m); (2) 
electric flux density D= G, t) (C/m?; (3) magnetic field strength H= i (7, t) (A/m); and 
(4) magnetic flux density B = h (7, t) (T). Mathematically, they are three-dimensional 
time-dependent vector fields, defined over unbounded spatial and temporal domains. 
However, in order to be represented on computers, their restrictions are used instead, 
defined on a so-called bounded “computational” domain ? € D c R°, and ona spec- 
ified bounded time interval O < t < T. A second set of two primitive physical quan- 
tities describes locally the EM state of the material objects: (5) volume charge density 
p = fs, t) (C/m3); and (6) volume current density J = fs, t) (A/m?) 

The derived global quantities, defined as spatial integrals on specified lines (open 
C, closed T), surfaces (open Sr, closed £), or domains (D>) of the local quantities pro- 
vide the global description of the EM field and objects: (1) electric voltage u = fe È- dř 
(V); (2) electric flux p = fs, D - dA (C); (3) magnetic voltage Uj, = fe H - d? (A); (4) mag- 
netic flux p = Ís È- dA (Wb); (5) electric charge q = h, p dv (C); and (6) electric current 
intensity i = hJ . dA (A). 

The global quantities are time-dependent scalar functions, associated with cor- 
responding one-, two-, or three-dimensional manifolds. The curves and surfaces con- 
sidered for modeling have to be sufficiently smooth (i. e., Lipschitz manifolds) and 
oriented. A closed curve T is oriented according to the right-hand rule as related to 
the supported surface, and a closed surface È is oriented outwards. Open lines or sur- 
faces are oriented arbitrarily. These definitions highlight that the local quantities are 
actually associated with differential forms in external calculus. Using a simplified lan- 
guage, we might say that the field strengths (E, H) are 1-forms, the fluxes (D, B, J) are 
2-forms, and the volume charge density p is a 3-form. A p-form is a quantity to be in- 
tegrated on a corresponding p-manifold (curve, surface, domain for p = 1, 2,3, respec- 
tively), by using a corresponding differential dx (elementary length dř, area dA, and 
volume dy, respectively). If we denote the space of p-forms with W”, then the primitive 
EM quantities are the p-forms: E, He wt, D, BJ ce W?, pe w°. 

The general laws of the EM field describe quantitatively the EM fundamental phe- 
nomena [33]. Their global forms, valid for any three-dimensional domain Dy and any 
surface Sr, with £ = dD; and T = Sp, are relationships between global primitive 
quantities, stated as follows: 

1. electric flux law (Gauss) 


Ws = 4p, (5.1) 
2. magnetic flux law (Gauss) 


Ps = 0, (5.2) 
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3. EM induction law (Faraday) 


dos, 
Seir 5.3 
ur at (5.3) 
4. magnetic circuit law (Ampere—Maxwell) 
d 
Umr = ts, + a ` 64) 
The local forms of (5.1)—(5.4) are called Maxwell’s equations: 
divD = p, V-D=p, (5.5) 
div B = 0, V-B=0, (5.6) 
>- OB >» oÈ 
1E=-—, VxE=-—, 5.7 
cur T x zf (5.7) 
. + ƏD sic d OD 
1H = —, H = —. ; 
cur J+ T Vx J+ a (5.8) 


These are general, fundamental, first-order PDEs of the macroscopic electromag- 
netism, valid only for media at rest, whereas (5.1)-(5.4) are valid also for moving 
media. Their proofs use the Gauss—Ostrogradsky and Stokes theorems and they can 
be written using the del operator V. 

On motionless surfaces of discontinuity, the local forms are 


div, D = p,, ñp - (D> - Dy) = pss (5.9) 
div, B = 0, ñp - (By - B,) = 0, (5.10) 
curl, È=0, Îi x (E, — È) =0, (5.11) 
curl, Ë = J; ñp x (Hy - È) = J, (5.12) 


where ñ, is the unit vector normal to the discontinuity surface, the subscripted vectors 
indicate their values at opposite sides of the interface, at the same point on it, p, = 
f,(7,t) = dq/dA (C/m2) is the surface charge density, and J; = Alt t) = tdi/dl is the 
surface current density. 

To ensure the completeness of the system of equations (5.1)—(5.4), which describe 
the EM field, three material-dependent constitutive relationships of the macroscopic 
electromagnetism are added [33]: 

1. Electric conduction law (Ohm): 


} =f,(B). (5.13) 


In conductors with affine characteristics J = oË + E;) or] =oE+ Ii 
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2. Polarization law: 
D = f (È). (5.14) 


In dielectrics with affine characteristics D = cE + Py 
3. Magnetization law: 


B = f(A). (5.15) 


In magnetic materials with affine characteristics B = ya + UoMp. 


For the particular case of linear media (È; = 6, P, = 6, M, 5 6), these relationships 
are linear functions described by the material coefficients: conductivity o (S/m), per- 
mittivity € (F/m), and permeability u (H/m), which are scalar quantities in isotropic 
media (function of position in nonhomogeneous ones and constant otherwise). In 
anisotropic media, they are second-order tensors, described by symmetrical and pos- 
itive definite 3 x 3 matrices. In affine media, the characteristics include, along with the 
linear term, a vector constant: the intrinsic electric field È; or intrinsic current density 
Ji = oE;, the permanent polarization È, and the permanent magnetization M,,, re- 
spectively. The affine modelis obtained through linearization of the nonlinear model, 
by using the truncated Taylor series expansion of the characteristic function around a 
given operating point (usually the origin), by retaining the first two terms only. Polar- 
ization P (C/m?) and magnetization M (A/m) are physical quantities describing the de- 
viation of the electric displacement and of the magnetic flux density in substance from 
their values in vacuum condition: P = D —e E, M = B/u — Ë. These quantities have a 
temporal component, dependent (linear or nonlinear) on the field strength, and a per- 
manent component (constant, independent of the field strength): P(E = P, (E )+ Pi, 
P,(0) = 6, PO) = Ë, M) = MB) + Mp, MiO) = 6, MO) = M,. 

In addition to the four general laws and the three constitutive laws, the macro- 
scopic electromagnetism includes two transfer laws which describe how the energy or 
the substance is transferred between an EM and other physical systems: 

1. The power transfer law (Joule) describes the energy transfer occurring between 
the EM field and conducting substances during the conduction process: 


p=J-E (W/m’), (5.16) 


where p is the power volume density transferred from field to substance. 
2. The mass transfer law (Faraday) describes the mass transfer in electrolytes 


=k] (kg/(m’s)), (5.17) 


where 6 is the mass density of the flux rate. 
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1V-D=p 
2.V-B=0 

3. V x É = -2B 
4. V x Ë = J+ 2È 
5 D=cE+B, 

6. B = pË + uoM, 
7. F=o(E+E;) 
8. p=J E 

9. =k] 


Figure 5.1: Diagram of electromagnetism relationships. Notations x’ and x” refer to distinct terms of 
the corresponding equation. 


The diagram shown in Figure 5.1 illustrates the causal relationships, as described by 
the nine laws of electromagnetism. This image highlights the fundamental EM phe- 
nomena and their intertwining, thus providing a physical meaning to the laws. The 
dotted arrows represent phenomena occurring only in time-varying regimes, whereas 
the thick, solid arrows represent phenomena occurring in both dynamic and station- 
ary regimes. 

A different graphic representation of fundamental relations of electromagnetism, 
this time from a mathematical perspective as opposed to the physical/causal view- 
point, is the Tonti diagram, also known as “Maxwell’s House” [9], as shown in Fig- 
ure 5.2 (left). The four pillars in this diagram are perfect De Rham sequences (Figure 5.2 


Figure 5.2: Left: Maxwell’s house. Right: De Rham sequence = chain containing spaces of differential 
forms, with increasing order. The sequence is perfect: the kernel of an operator is in the domain of 
the previous operator. 
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(right)), each one being associated with an EM field primitive quantity (E, D, B, H). The 
arrows representing Maxwell’s relations are emphasized. 
On the basis of these nine laws, two important theorems can be proved [33]. 
1. The electric charge conservation theorem highlights the strong link existent be- 
tween charge and current: 
dqp, 


iy SS dt Y Ds. (5.18) 


2. The EM energy conservation theorem has the following form in motionless media: 


YD; (5.19) 


Ps = Pop, $ dt 
where P; = — h(E xĤ)-ñ dA is the power transferred inward through the boundary 
x = oD; of the considered domain, Pep, = f» pdv is the power transferred to 
= 
the conducting substance inside the domain, and, in the case of linear media, 
Wp, = f» (È . D +H - B) /2 dv is the EM energy inside the domain. 
2 


The fundamental problem of EM field analysis consists of the computation of the EM 

field, as a solution to Maxwell’s equations, and of the constitutive relations for mate- 

rials. In the general transient case, this problem can be formulated as follows. Given 

- the shape and dimensions of the computing domain Dy; 

— thevalues of g, €, pin each point of Dy or these material constants in each assumed 
linear or affine and homogenous subdomain; 

— the internal field sources at each point of the computing domain, provided by E;, 
P, and My vectors; 

— boundary conditions: at each moment in time O < t < T, at each point on the 
domain’s boundary È = ðD; the tangential component, either of the electric field 
È, or of the magnetic field Ē;; 

- initial conditions: at each point in Dy the electric flux density D and the magnetic 
flux density Bat the initial moment t = O, with div BC, 0) =0. 


Find the solution represented by the fields E ; D, B, H Ps J, defined on the computing 
domain D and on the time interval 0 < t < T. 

Such a PDE problem is mathematically correctly formulated (“well-posed” in the 
Hadamard sense) if a solution exists, is unique, and is continuously dependent on the 
problem data (more precise, if the problem is well-conditioned). The mathematical 
formulation of the field problem requires a selection of the most adequate functional 
framework and the reformulation of the EM field problem within that framework. 

The theorem of the solution uniqueness states that ifo > 0,€ > 0, u > O, the 
solution of the Maxwell equations (5.1)—(5.4) together with the constitutive relations 
in affine media (5.13)—(5.15) is unique if the following conditions are given: 
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- (CD): internal sources of the field in domain D;: 
ÈE, t) B E, t), ME, t), Pe Dy, O<t<T, (5.20) 
- (C2): boundary conditions on È: 
AXxE(?,t)xn=E,, Pe Sp CDs, (5.21) 
ax HA(?,t)xni=H,, ?eSg=03Dz-Sp O<t<T, (5.22) 
— (CO): initial conditions: 


DF 


© 
© 
Il 
(amts 
= 
= 
~) 
© 
[vni 
~ 
m= 
oO 
a 
Il 
eol 
o 
= 
Q 
YS 


7 € Dy. (5.23) 


The proof is based on the EM energy theorem and on the null-solution lemma, which 
states that a linear equation has a unique solution if the associated homogenous equa- 
tion (i. e., a field problem with null uniqueness conditions) has only the null solution. 

Most often, the boundary conditions are null. The condition È, = 6, called per- 
fect electric conductor, relates to a perfect conductor (superconductor with 1/0 = 0) 
located on the boundary. 

If, to facilitate the understanding, we assume that E ; is given over the entire 
boundary, then the result of the field analysis is a unique distribution of the EM field 
in the computing domain. This result is uniquely determined including the com- 
ponent H, on the boundary. Thus, a correct formulation of the problem of EM field 
analysis guarantees a proper definition of an operator which links the two input and 
output vector fields È, and Hi, respectively, defined on the boundary = = dD; and 
on the time interval 0 < t < T. This transfer operator defines a dynamic input-output 
(I/O) system, which describes the response of the entire domain D; under several 
excitations. 

Although from the perspective of system theory the transfer function refers only 
to the boundary quantities, treating the studied object as a black box, in fact the trans- 
fer function depends on the internal structure and material behavior. Thus, to extract 
this transfer operator the fundamental EM field problem has to be solved in the entire 
Dy. If the medium of this domain is linear, then this I/O system is also linear and the 
I/O transfer operator is a linear one, since the solved equations are linear. From the 
perspective of system theory, this I/O dynamical system has the Maxwell equations as 
state equations. Its state variables (those describing the initial conditions) are the elec- 
tric and magnetic flux densities D, B in the entire domain Ds. The I/O system thus de- 
fined, which is an EM system, has input and output signals, as well as state variables, 
all of infinite dimensions, as they are functions defined on x and D,, respectively. 

These mathematical objects have only theoretical value, since in practice, in the 
computer representations their finite approximations are used. For example, in the 
case of uniform cubic grid with n nodes along every direction, the system has 2 com- 
ponents x 6 faces x n? nodes = 12n° I/O signals and 3 components x 2 vectors x n? nodes 
= 6n? state variables. These numbers of order O(n’), O(n?) tend towards infinity when 
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n — oo. It can be said that this EM field system is of “infinite-inputs, infinite-outputs” 
(IIO) type. It would be multiple-inputs multiple-outputs (MIMO) if the number of in- 
put and output signals would be greater than one, but finite. Such a MIMO EM system 
is discussed in the next section. 

In the classical system theory, when they have a finite number of state variables, 
these kinds of systems are finite, with lumped parameters. In the EM field systems, 
since they have an infinite number of state variables, distributed in space, we say that 
these systems are with distributed parameters. 


5.3 Coupled field-circuit problems and EMCE 
boundary conditions 


In this section we consider the reduction of an EM system from an infinite number 
of I/O signals to one with a finite number of I/O signals. This approach is necessary 
when EM devices with distributed parameters (which include field effects such as eddy 
currents) are coupled with external circuits. 

By definition, electric circuits have a finite number of components interconnected 
at nodes, characterized by their electric potentials. A subcircuit with m terminals is 
characterized at each terminal node by a pair of scalar quantities: current and poten- 
tial. Depending on the excitation mode, one of these quantities is the input signal, the 
other one being the output. Consequently, an m-polar circuit is an I/O dynamic system 
of MIMO type, with m - 1 inputs and m - 1 outputs, the m-th node being the reference 
potential. 

In order to couple circuits and EM devices, the latter need boundary conditions 
compatible with external circuits. Specifically, on the boundary of the EM device 
model, a finite number m of equipotential patches represent the terminals connected 
to the nodes of the exterior circuit. Usually, these terminals are very good conductors 
(copper, silver, or even gold-plated), which may be modeled as perfect conductive 
parts without a significant variation of electric potential over their surfaces. More- 
over, the boundary conditions need to ensure that for each terminal the associated 
potential and current can be correctly defined. 

The multipolar electric circuit element (ECE) is a domain D (Figure 5.3) with bound- 
ary conditions that ensure the compatibility with the ECEs [42, 31]. 

The boundary È = ðD of the domain comprises m disjoint parts S4, S2, . . . Sm, with 
S = Uk Sx, called electric terminals on which: 


ni-curlE(P,t)=0 (V)Pe%, (5.24) 
ni-curlH(P,t)=0 (V)Pex-S, (5.25) 
nx E(P,t)=0, (V) Pes, (5.26) 


where ñ is the normal unitary vector in P. 
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Figure 5.3: The multipolar electric circuit element (ECE). 


These boundary conditions ensure: the absence of magnetic coupling with the exte- 
rior (5.24); electric coupling only through terminals (5.25); and equipotentiality of each 
terminal (5.26). The electric current of a terminal k is defined as the total current (con- 
duction and displacement) flowing through it: i,(t) = fr, H - dř, where T% = 0S; is 
the boundary of the terminal surface S,. We assume that T, are oriented so that the 
associated normal of S, is inwards oriented. Due to (5.25), the sum of all terminal cur- 
rents is zero and the Kirchhoff current law is a consequence. The electric voltage of the 
terminal k is defined as the integral v,(t) = fa, E . dř along an arbitrary curve Cx in- 
cluded in x, which is a path between a point on Są and a point on a reference terminal, 
say, S„. Condition (5.24) ensures the consistent definition of the terminal voltage, its 
independence of the shape of Cp, and the Kirchhoff voltage law as a consequence. 

A uniqueness theorem can be stated and the power transferred by any ECE 
through its boundary, from outside to inside, is given by 


P=- ge x H)-dA = p x H) - (dA) = py curlH) - (-dA) 


m m m-1 
= Vx > [omā (-dA) = vk = fä -dř = $ Vgik. (5.27) 
kg, KEIR k=1 


If the terminals are excited by known potentials, then the problem of EM field 
analysis in a linear domain with ECE boundary conditions has a unique solution as it 
can be probed by using the lemma of the null solution, 

Consequently, the terminals’ currents are output signals univocally defined by 
solving the field problem. As the domain is linear, the equations are linear, and the 
device with ECE boundary conditions is a linear, MIMO dynamic system with m - 1 in- 
put signals and m - 1 outputs. The field problem is also well formulated, if only p < m 
terminals have known voltages, one is grounded, and the remaining m — p — 1 have 
known currents. This is the case of the hybrid excitation. The ECE is voltage-excited 
if p = m - 1 and current-excited if p = 0. Under null initial conditions, due to the lin- 
earity of the field equations, and by applying the Laplace transform, the output vector 
v(s) = [V,(S), V2(S),.--3 Vin_1(S)] T of terminal potentials of a current-excited ECE will be 
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linearly dependent on the input current vector i(s) = [i,(s),i,(s),...; ats) 
v(s) = Z(s)i(s), (5.28) 


where Z(s) is the operational impedances matrix. Since the field problem has a state 
space of infinite dimension, it is expected that the operational impedances have an 
infinite number of poles, although it is a finite (m — 1) x (m — 1) matrix. In addition, 
it is expected that the complex impedances matrix, resulting from replacement of the 
complex frequency s with jw, will be positive real, since the complex power S has the 
real part P positive for any excitation since 


S=P+jQ=) vi =i" v= i"Zi, (5.29) 


where S = - 6, (Ë x H )-dA, P = fas oE-E dv, Q = w fp (BH - Ë. D“) dv. Since 
the material constants are positive scalars, the ECE device is reciprocal (with a sym- 
metrical impedance matrix) and passive (with absorbed active power P > 0, regard- 
less of the field distribution). When terminals are voltage-excited, the complex admit- 
tance Y = Z™ matrix is defined similarly. In general, some terminals can be current- 
controlled, and the others voltage-controlled, in which case the device is character- 
ized by a hybrid operational matrix H. 

The generalization of the ECE concept is the multipolar EM circuit element 
(EMCE), which has not only electric terminals, but also magnetic terminals. The 
power transferred by an EMCE with n electric terminals and m magnetic terminals is 


$ S : n-1 ; m-1 d Pk 
P= -QË xñ). da = > Yale + È mat” (5.30) 
= = 

where um, is the magnetic voltage of the magnetic terminal k and øx is the magnetic 
flux of flowing through the magnetic terminal k. This expression is completely com- 
patible with the power transferred by a multipolar electric circuit connected to the 
electric terminals and a multipolar magnetic circuit connected to the magnetic termi- 
nals of the EMCE. 

We used EMCE formulation in applications such as modeling RF passive compo- 
nents or blocks as in [12], RF models of microelectromechanical switches [14, 39], or 
even modeling of myelinated axonal compartments [27]. Other researchers also used 
this formulation in magnetoquasi-static (MQS) problems for inductance extraction 
[38]. Similar conditions, although with a different definition for the terminal voltages, 
are proposed in [23]. 

In what follows we will assume a simple connected domain. The case of multiply 
connected domains have been addressed in the early paper of Timotin in [52]. The very 
recent comprehensive study of Hiptmair and Ostrowski [24] proves the usefulness of 
these boundary conditions, currently not available in popular FEM software. 
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5.4 Simplified models for spatial distributions and 
transmission lines 


The next step is to reduce the dimension of the state space of the EM system. In the 
general case, the complete description of the EM field state requires six scalar quanti- 
ties (components of D and Ë) at every point of the physical three-dimensional space, 
at any moment. A dramatic reduction, to four or even two scalar quantities per point, 
can be achieved if a space simplification to two dimensions or even one dimension is 
possible. 

Transmission lines (TLs) are plan-parallel structures which guide waves, encoun- 
tered for instance in microelectronics, as shown in Chapter 4 of this volume [5]. At low 
frequencies the transmission is mainly due to conduction, whereas at high frequen- 
cies it is much more due to EM induction and displacement current. The aim of this 
section is to define the I/O dynamic system with distributed parameters associated to 
multiconductor TLs. 

In the simple case of a two-conductor line, the electric field has a two-dimensional 
transverse distribution similar to that of an electrostatic (ES) field, being determined 
by the conductors’ potentials, with field lines normal to the surfaces of the conductors. 
At high frequencies, when the penetration depth of the EM field can be neglected, 
the current is distributed on the conductror’s surfaces and the magnetic field is also 
transverse and perpendicular to the electric field (Figure 5.4). Under these conditions 
the current i(z, t) carried by the two conductors in opposite directions and the voltage 
v(z,t) between the two conductors completely describe the EM field distribution for 
any cross-section of the TLs. That is why these two scalar time-dependent functions 
are selected as state variables. 

The TL equations, derived by Heaviside in 1880, can be obtained either starting 
from Maxwell’s equations, splitting the phenomena according to transverse and longi- 
tudinal operators as below, or by using an approach typical to circuits with distributed 


Figure 5.4: Electric (left) and magnetic (right) fields for a two-conductor transmission line. 
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Figure 5.5: TL as an EM device (left) and its circuit model with distributed parameters (right). 


parameters (Figure 5.5). The line is characterized by the per unit length (p. u. 1.) param- 
eters: R (Q/m), L (H/m), C (F/m), and G (S/m), which may be extracted by solving four 
field problems in the electroconductive stationary (EC), magnetostationary (MG), ES, 
and EC regimes, respectively. In the simplest case of a homogeneous dielectric, only 
one two-dimensional ES field problem has to be solved, since G = Co/e and L = Cy/e. 

TLs are multipolar elements of electric circuit, their modeling being essential in 
many RF applications. 

In the case of a multiconductor line with n conductors, 2n scalar time-dependent 
signals are required at each space point along the line. For each conductor the state 
variables are the local current and potential. By solving the PDE equations of the EM 
field, the equations which describe the propagation along an n-conductor TL are ob- 
tained [16]: 


t 
OV; _ 10; n 0 Oim re) [Se p 
-z Hit 5 (ne x | T) in Ddr > 


m=1 


di 2 Ov 
= ka $ (Sinn + Chm } (5.31) 


m=1 


where r° is the p. u. 1. DC resistance of the conductor k, I „ are p. u. 1. external induc- 
tances (self inductances for k = m and mutual inductances for k + m) of the conduc- 
tors k and m when the return current is distributed on the surface of the substrate, and 
lim are “transient p. u. l. inductances,” defined as averaged values on the cross-section 
of the conductor k of the vector potential obtained in zero initial conditions by a unity 
step current injected in conductor m. 

For current and voltage null initial conditions, the Laplace transform of the gen- 
eral propagation equations (5.31) are 


„mes = Z(s)i(z, s), 
di(z,s) _ 
a Y(s)u(z, s), (5.32) 


with O < z < l, where lis the line length and v and i are n-dimensional vectors of 
voltages and currents, respectively. They are similar to the TL equations (also named 
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telegrapher’s equations) in the frequency domain, obtained from Kirchhoff equations 
for RLCG lines with distributed parameters. The only major difference is that here resis- 
tances and inductances are dependent on the complex frequency s in order to model 
the skin effect in the line’s conductors and the eddy current losses in substrate. To ex- 
tract the frequency dependence of inductances, an EM field problem in MQS regime 
has to be solved. The field distribution inside the conductors inserts new state vari- 
ables (an infinity, if their values are considered for each z). In [28] a method named of 
the two fields is proposed, applied to extract the frequency dependent p. u.l. param- 
eters (admittance Y and impedance Z) for lossy multiconductor transmission lines, 
from the field solutions. Based on them a method to extract a parametric reduced- 
order model for this system is generated. Another approach to include in the TL model 
the eddy currents effect is based on the substitution of the longitudinal p. u. 1. resis- 
tance with the approximate ladder model developed in Section 5.6. 
If we denote 


Tr ow 


then (5.32) can be written as 
= Au, (5.34) 
and its solution is 

u(z,s) = exp(Az)u(0,s). (5.35) 


If the quantities of the line ends are denoted by 


_[ v ]_[ vs) Ty] v(l, s) 
=r j- [wsh el i, Flas ee?) 
then 
u,=Tu_, where T = exp(Al) =| Ta, t | (5.37) 
Ta Tx 


The terminal operational admittance matrix Y can be computed from T as 


: -1 -1 
| i | a y| ka f Y= | Toli T | (5.38) 
1, Vi TTT -Ta -TT 


Here, the terminals are considered voltage-controlled and, therefore, the multicon- 
ductor line with n conductors is a MIMO-type system with 2n input signals (potentials 
of the n close and n distant terminals) and 2n output signals (currents through these 
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terminals). However, being a distributed system (with an infinite number of state vari- 
ables, one-dimensionally distributed along the line 0 < z < J), the dimension of state 
space is infinite. Actually, the multiconductor line meets the ECE boundary conditions 
(5.24)—(5.26) for an element with 2n terminals that are the close and distant extremities 
of the n conductors. The EM field within the TL is distributed in space and time, but 
in harmonic state, the information is concentrated in a point in the frequency domain 
and its one-dimensional space distribution is given by one complex number (voltage 
or current) related to each line terminal. 

For the particular case of a line with a single conductor over the ground, which 
is a two-port quadrupole described by a system with distributed parameters with two 
input and two output signals, the global admittance has the following expression: 


Y. Y. cosh(yl) 1 
= 11 12 = Z, sinh(yl) Z, sinh(yl) _yl 
Y= | Sg | 2 | WD Z | _ yT, (5.39) 
2 -2 Z.sinh(y) Z, sinh) 


where Z, = V(R + sL)/(G + sC), y = V(R + sL)(G + sC) are the characteristic impedance 
and the complex propagation constant, respectively, depending on p. u. l. parameters 
R, L, C, and G and frequency. The result of the presented approach related to this par- 
ticular case has an analytic expression, and thus illustrates the AAM approach. 

The TL approach is essential for the MOR of on-chip interconnect lines (see also 
Chapter 4 of this volume [5]). Although geometric reduction is very effective, this ap- 
proach is not the final step in the complexity reduction; it needs to be sustained by an 
additional order reduction process, in order to obtain the desired reduction to a mini- 
mal finite order instead of an infinite order [30]. This section uncovered only one step 
in the process of complexity reduction of EM systems, that based on simplifications 
of geometrical structure, in which for adequate geometrical modeling it is assumed 
that each component of the EM field is dependent only on certain coordinates from a 
proper selected coordinates system. This choice, called geometrical modeling, reduces 
the complexity of particular EM systems in a dramatic manner. 


5.5 Regimes of the electromagnetic field and models 
with lumped parameters 


The complexity reduction of general EM systems can be done by simplifying the 
physics, i.e., by disregarding some phenomena. This can be acceptable under certain 
conditions, specific to each particular studied case. In practice such simplifications, 
called a field regime, are based on a series of hypotheses so that the theory remains 
coherently and rigorously mathematically formulated. This is the main advantage of 
using a field regime, even if the obtained model does not perfectly reflect the reality. 
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The EM field regime described by Maxwell’s equations (5.5)—(5.8) and (5.13)—(5.15) 
is known as the general electrodynamic (ED) regime or the full-wave regime. 

Since the magnetic flux density B is divergence-free, this field is solenoidal and 
thus a magnetic vector potential A can be defined so that 


B=VxA. (5.40) 


In order to define a unique A a gauge condition is needed, which imposes a constraint 
on the divergence of A. The simplest condition is the Coulomb gauge: V - A = 0. By 
substituting (5.40) in the EM induction law (5.7) it follows that 


> OA 3 
E+— )=0. 5.41 
Vx ( + at ) ( ) 


=-VWV = EF=-yW-—. (5.42) 


The scalar potential V and the magnetic vector potential A are called electrody- 
namic potentials in the ED regime of the EM field. In terms of differentials, with a sim- 
plified language, the scalar potential is a 0-form V € W° and the vector potential is a 
1-form A € W!. The perfect character of the De Rham sequence (Figure 5.2) guarantees 
the existence of these two potentials. 

In linear media with imposed current J, Maxwell equations conduce to second- 
order PDEs, expressed in potentials which are state variables [33]: 


=> 2 D 3 aA vA V ? 
VLA =J Fs vx(wxA)roS +255 +oW teva =], (5.43) 
> op op 3 
Tane $ V)= Æ J, ; 
VJ 7 >V. (oVV) aa J; (5.44) 


where v = 1/y and the gauge V - (aA) = 0 was used. Their solution is an EM field that 
propagates with a finite speed. 

A complexity reduction of general EM systems can be achieved by eliminating 
some quantities that are irrelevant in certain conditions. For example, if the EM field 
varies sufficiently slow so that wave propagation can be ignored, one of the two field 
regimes known as quasi-static can be used [8]. 

The electroquasi-static (EQS) regime is intended especially for the study of capac- 
itive effects, where the phenomenon of EM induction is neglected (Figure 5.6). For- 
mally, u = O and thus B = 0. This choice reduces by half the dimension of the state 
space, since the only state quantity is D. An example of usage is an RCG transmis- 
sion line (with null line inductance), used to model one-dimensionally distributed 
resistive-capacitive combined effects. 
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Figure 5.6: Causal diagram of the EQS regime. 
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Figure 5.7: Causal diagram of the MQS regime. 


The MQS regime is intended especially for the study of inductive effects, where the 
displacement current is neglected (Figure 5.7). Formally, € = 0 and thus D = 0. This 
choice reduces by half the dimension of the state space, since the only state quantity 
is B. An example of usage is an RLG transmission line (with null line capacitance), 
used to model one-dimensionally distributed resistive-inductive combined effects. 


The electrostatic regime, capacitance extraction 

A very efficient method to reduce the complexity of EM systems is by modeling them 
with circuits having lumped elements (resistors, inductors, capacitors). In these mod- 
els, the current and the electric and magnetic fields are segregated, and thus their en- 
ergies are concentrated: electric energy in capacitors, magnetic energy in inductors, 
whereas the resistors do not concentrate EM energy. The fields interact only through 
the circuit. 

The extraction of the corresponding lumped parameters (R, L, C) is carried out by 
assuming stationary field distributions, even if the EM field is variable in time and, 
consequently, the circuit will be simulated in a dynamic regime. Thus, the ES regime 
is used to extract capacities (C), the MG regime for extracting inductances (L), and the 
EC regime for extracting resistances (R). 
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Figure 5.8: Causal diagram of the ES regime. 


In a static regime there is no time variation, no motion, and no energy transfer, so 
that power and currents are zero. The causal diagram and the first-order fundamental 
differential ES equations are shown in Figure 5.8. The nonrotational character of the 
electric field allows definition of the electrostatic scalar potential 


E=-vV, (5.45) 


which is the solution of the elliptical second-order PDE of Poisson type, with a div-grad 
operator [33]: 


- V- (EVV) = pp (5.46) 


where p; = p-V- È,. In homogeneous media, without charges and permanent polar- 
ization, the equation becomes of Laplace type, AV = 0. 

On discontinuity surfaces between two bodies with different dielectric character- 
istics, the interface conditions are 


3 3- 3 dV, dV. m > > 
ñn : D2- Dy) = ps > & T & n = Ps — Ön - (Py, — Py,): (5.47) 


ñp x (Ey - È) = 0 = V; = Vo. (5.48) 


These interface conditions highlight the continuity of potential due to the continuity 
of the tangential component of the electric field strength and the continuity of the 
normal component of the electric induction when there is no permanent polarization 
and the interface is not charged. 

The boundary conditions that ensure the potential uniqueness are [8] 


Dirichlet: V(P) =fp(P), PeSp#9, Sp cX=OD, (5.49) 
Neumann: = =fy(P), PeSpy=È}-S5p. (5.50) 


The capacitive interaction between n + 1 conductors located in a linear dielectric 
(Figure 5.9 (left)) is described by Maxwell’s equations for capacitances: 


qı Cu Cy >o Cin vı 
d2 Cy Ca ` Ca v2 


Il 


e q=CV, (5.51) 


Qn Cm Cm `t Cnn Vn 
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Figure 5.9: Capacitance extraction. Left: Excitation of conductors needed in the ES problem from 
which column j of the nodal capacitance matrix is computed. Right: Equivalent circuit forn = 3. 
Cy are partial capacitances that are computed from the nodal capacitances. 


where q,V e€ R” are the vectors of charges and potentials, respectively, and C = 
(Cy)ij-in E R™" is the matrix of conductors’ capacitance, also named matrix of nodal 
capacitances. The column j of the matrix C is extracted from the solution of a funda- 
mental ES problem in the dielectric, with Dirichlet boundary conditions, null on the 
surface of all the conductors except for conductor j for which V = V; (Figure 5.9 (left)). 
The capacitance C = qx/Vjly,-o14j, Where k = 1,n is computed from the charge 


a = QD- dA =- Qed, (5.52) 


which is linearly dependent on V;. The computation of the matrix C needs the solving 
of n distinct ES field problems, obtained by exciting successively only one conductor, 
the other conductors being grounded. 

Due to passivity and reciprocity, C is symmetrical, positively defined, and diag- 
onal dominant. Its inverse S = C7 is called the matrix of potential coefficients. The 
electric energy stored in this system is 


1 
W, = 
a) 


viq= sv'ev = sav = 5a'sq > 0. (5.53) 


Relation (5.51) can be modeled with a capacitive circuit, having the topology of a com- 
plete polygon (Figure 5.9 (right)). By applying the nodal method, it follows that the 
capacitances of this circuit (which have positive values), called partial capacitances, 
can be computed from the nodal capacitances as 


Chj = CK Vkj = 1,...,n, k +j, (5.54) 


n 
C=} Ge Yk=1...n. (5.55) 
j=1 


This procedure is widely applied in practice to extract parasitic RC parameters 
in integrated circuits. The ES regime results provide the C parameters, whereas the 
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EC regime is similar to ES. If € is replaced with o, then the conductance values are 
obtained. 

The advantage of having a model described as an equivalent capacitive circuit 
comes from the fact that its complexity reduction can be efficiently achieved by re- 
moving negligible equivalent capacitances connected between floating nodes (e. g., 
capacitances smaller than a threshold of 0.1% relative to capacitances to ground). 
This leads to a sparse nodal capacitance matrix, which does not lose its symmetry and 
positive definite, thus passive, characteristic. 

Since the capacitance between two conductors is inversely proportional with the 
distance between them, it follows that around each conductor there is a spherical 
“window” containing coupled conductors. Outside this window, the capacitive cou- 
pling may be ignored. This observation simplifies the ES field problem that has to be 
solved to extract a (sparse) column of the nodal capacitances matrix. Even more effi- 
cient methods can be conceived if the pattern of the sparse nodal capacitance matrix 
is known, e.g., it is banded [35]. Such strategies are important and compulsory for 
integrated circuits, where the number of conductors is huge, over one million. 


The magnetic stationary regime, inductance extraction 

In a stationary regime there is no time variation and no motion, but energy transfer 
is allowed, so that power and currents are nonzero. The causal diagram and the first- 
order fundamental differential equations of the magnetic stationary (MG) regime are 
shown in Figure 5.10. In the particular case where there are no currents, this becomes 
the magnetostatic (MS) regime. The rotational character of H does not allow a natural 
definition of a scalar potential, this being the main difference between MG and MS. 
The solenoidal character of B allows however the definition of the magnetic vector 
potential 


B=VxA, (5.56) 
which is the solution of the second-order PDE with a curl-curl operator [33]: 
Vx (vV x A) =I, (5.57) 


where i; =J+Vx (vB,). As discussed in the general ED case, a gauge condition is 
required to ensure the uniqueness of the magnetic vector potential. In homogeneous 


oy 


P 


Figure 5.10: Causal diagram of the MG regime. 
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media, with Coulomb gauge V -Å = 0, the magnetic vector potential satisfies a Poisson 
vector equation:' 


-AÅ = y], (5.58) 


If there are no sources, i. e., J: = 6, then the equation becomes Laplace, AA = 0. 
On discontinuity surfaces between two bodies with different magnetic linear char- 
acteristics, the interface conditions are 


ñp x (Ay - Ay) = J, > ñp x (WV x Ay -v,V x A,) = Jẹ (5.59) 
ñp- (By - B,) = 0 > yy (V x A, - V x Å) = 0. (5.60) 


If there is no current sheet at the interface, from (5.59) it follows that H,, = H,,, and 
if v; = v, then (V x A,), = (V x A,),. From the Coulomb gauge it can be derived that 
Am = Am and from (5.60) Å} = Å». Consequently, the magnetic vector potential is 
continuous, A, = A,, across interfaces. 

An important practical problem is the quantitative characterization of the induc- 
tive interaction between n conductive wires. This interaction is described by Maxwell’s 
equations for inductivities, which give the magnetic fluxes produced by these n con- 
ductors placed in a linear magnetic medium: 


Pı La Ly o Ln iy 
P2 Ln Ly > Ln iy 7 

i = $ l e gp=li, (5.61) 
Pn Lm Lig cat Lan in 


where @,i <€ R” are the vectors of magnetic fluxes and currents, respectively, and 
L = (Lj)ij-1n € R”” is the matrix of inductances, holding self- and mutual inductances. 
The column j of the matrix L is extracted from the solution of a fundamental MG prob- 
lem in the magnetic medium, where all the currents are zero, except for conductor j 
for which i; + 0. The mutual inductance between wires k and j is Ly = Qx/ijli-o,14)> 
where k = 1,n is computed from the magnetic flux 


P= | B-da = A-ar, (5.62) 
Tk 


St, 


which is linearly dependent on i. 

The computation of the matrix L needs the solving of n distinct MG field problems, 
obtained by current exciting successively only one conductor, the other conductors 
being open. 


1 Vx(VxA) = V(V-A) — AA = — AA, due to the Coulomb gauge. 
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Due to passivity and reciprocity, the matrix L is symmetrical, positively defined, 
and diagonal dominant. Its inverse is denoted by K = L™!. The magnetic energy stored 
in this system is 


lirj, 1 7, DE 
51 Li 59 i= 39 Kọ > 0. (5.63) 
In the case of wire conductors, (5.62) can be applied only for the computation of mu- 
tual inductances, since the integral is not convergent for self-inductances. The self- 
inductances can be computed from (5.63), where W,, = fe Å -J dv. 

Relation (5.61) can be modeled with an inductive circuit with n coupled coils. 

The modeling of inductive effects in integrated circuits is becoming increasingly 
important as their clock speeds increase. However, on-chip wiring is a large structure, 
which requires without doubt complexity reduction ofits models. The inductance ma- 
trix is a full one, and its size is huge, in particular if wire segmentation is applied 
as in partial electric equivalent circuits [47]. The complexity reduction of the equiv- 
alent circuit and the matrix sparsification is more difficult for inductances than for 
capacitances. Sparsification of a dense or even full matrix means their approximation 
with a sparse matrix, which has very few nonzero elements. One straight approach to 
make the inductance matrix sparse is simply to discard those mutual coupling terms 
of L, which are below a certain threshold. This approach, however, does not guar- 
antee the positive definiteness of the resulting inductance matrix and therefore the 
model passivity is not preserved. A series of alternative methods for robust sparsifica- 
tion of the inductance matrix which preserve the passivity were proposed, such as the 
K-method [34], the vector potential equivalent circuit method [54], and the magneto- 
electric equivalent circuit method [29]. 


lit 
W, i 
m= 5 p 


Reduced circuit models with lumped parameters 

The simplest model of a system with n conductors in an ED regime, in which there are 
EM induction phenomena, and capacitive effects are neglected, can be modeled by an 
n-port circuit (Figure 5.11 (left)) characterized by the equations 


ig LŠ +Ri, (5.64) 


where R = diag(Rj,...,R,) € IR", with Rẹ being the resistance of the conductor k; 
i,u € R” are the vectors of currents and voltages along the wires, respectively. These 
matrices are extracted from the MG and EC field solutions. The number of state vari- 
ables is equal to the number of inductors, i. e., n. 

Similarly, the simplest model of a system with n + 1 conductors in an ED regime 
placed in an imperfect insulator, but where the EM induction phenomena are ne- 
glected, can be modeled by a multiport circuit (Figure 5.11 (right)), characterized by 
the equations 


i= cf GY: (5.65) 
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Figure 5.11: The simplest circuit model for n conductors in MQS (left — RL circuit) and EQS (right - GC 
circuit). 


where C is the capacitance matrix and G is the matrix of conductances for resistive 
losses in the dielectric (G = Co/e for homogeneous media). These matrices are ex- 
tracted from the ES and EC field solutions. The number of state variables is also equal 
to n, being less than the number of capacitors since they are in excess, as can be easily 
seen (there are loops consisting solely of capacitors). 

In the two models discussed above, which can be used in MQS or EQS regimes, 
the lumped parameters are extracted from static or stationary regimes. The results 
they give may be acceptable at some frequency ranges, but they might not be accurate 
enough at high frequencies, e. g., when eddy currents and skin effects become rele- 
vant. As will be explained in the next section, in this case circuit models with lumped 
parameters extracted from MQS and EQS fields can be used. Even so, neither these 
models are accurate enough beyond a certain frequency, where both inductive and 
capacitive effects have to be considered simultaneously. 

It is evident that both inductive and capacitive effects are considered if the field 
regime used for extraction is the general ED. However, combined inductive and capac- 
itive effects can be more easily modeled if the RL and CG circuits above are combined 
in T or II models, without solving the ED field problem (Figure 5.12). These models 
have the order of complexity equal to 3n. 


Uk Uk 


> > 
; R;/2 Lien /2 R,/2 Lin /2 i Ry Dkk 
$ INNA 4 4 N > NMa e 
° ° e 
on Gkk — Cex Lry/2 Grx/2 ZZ Ork/2 Res Grk/2 Z COrr/2 


Figure 5.12: The simplest RLC circuit model for conductor k, where k = 1,...,n: T (left) and N (right). 
To simplify the figure, partial capacitances between conductors are not shown. 
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Even if they have inductive, capacitive, and conductive effects, the T or II models are 
not able to describe the field propagation. If this effect is important, in the case of 
wire-shaped conductors, the line of length 1 may be decomposed in p segments. If 
each segment has a length I/p « A, where A is the wavelength (the rule of thumb is 
p = 4l/A), then the propagation along a segment can be neglected. Consequently, each 
segment can be modeled with a T or II schematic. Then, by connecting all segments’ 
models, a reduced system with (2p + 1)n state variables is obtained. 

The great advantage of modeling with lumped parameters is that the original field 
problem (EQS, MQS, or ED) with an infinite state space is reduced to an I/O dynamical 
system with a finite number (from n up to (2p + 1)n) of state variables. Moreover, the 
number of I/O signals is also finite (2n inputs and 2n outputs for n conductors). In 
three-dimensional devices, all three directions have to be “segmented,” but this is 
carried out with numerical approaches, e. g., as discussed in Section 5.7. 


Identification of the appropriate field regime 

The discussion above revealed that the complexity reduction or EM system can be 
achieved by choosing an appropriate field regime. For this, the characteristic times, 
which depend on the material constants £, 4, o and the characteristic length l of the 
device under study, are computed and compared [44]. 

There are three characteristic times of an EM device: (1) charge relaxation time 
Te = €/o is the relaxation time of an electric field in a conductor; (2) magnetic field 
diffusion time T,, = vol’ is the diffusion time of the magnetic field in a conductor; and 
(3) propagation time Tem = l/c, with c = 1/ yZ}, is the time in which a wave propagating 
with speed c travels a distance l. 

It follows that T2 = TeTm- In very good conductors Te < Tem X Tm, i. €., Charge 
relaxation time is negligible and magnetic inductive effects are preponderant. In poor 
conductors, Tm « Tem < Te, the time for magnetic field diffusion is insignificant, and 
the charge relaxation is important. 

If we denote by t the characteristic time, defined as the duration period or time 
constant of the phenomenon under study, then the appropriate regime can be decided 
by comparing T with Tem» To, Tm. The graphical representation shown in Figure 5.13 
illustrates this comparison. A point in this diagram has as abscissa the decimal loga- 
rithm of T/Tem and the ordinate the decimal logarithm of T,,/T em. 

If Te < Tm (i. €., Te < Tem < Tm), then the corresponding point on the map will be 
in the first “quadrant” of the map (x > 1, y > 1), and if T < Tm, an MQS regime has to 
be considered (e. g., the case of very good conductors), otherwise a static/stationary 
magnetic regime is appropriate. 

If Tm < Te (i.€., Tm < Tem < Te), then the corresponding point on the map will 
be in the fourth “quadrant” of the map (x > 1, y < 1), and if T < Tẹ, an EQS regime 
has to be considered (e. g., the case of poor conductors), otherwise a static/stationary 
electric regime is appropriate. The lines draw on the map are not very strict. In fact, 
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Tx 10 tom T > 10Tem ar Figure 5.13: Map of EM field regimes, as a 


function of characteristic times. 


if the phenomena are very slow (T > Te > Tm OT T > Tm > Te), then the station- 
ary/static regimes will be definitely used for modeling; otherwise (for intermediary 
speeds) quasi-stationary regimes may be chosen. In the case of fast phenomena, there 
is a small band region on the left side of the x = 1 axis, where both charge relaxation 
and EM induction phenomena are considered, but the propagation is ignored since 
the displacement current is neglected. This is called the EM quasi-stationary (EMQS) 
regime. 

When the phenomena are extremely fast (T < Tem, usually T < 10T,, is assumed, 
i. e., the left semi-space with respect to the x = 1 line), then the wavelength is much 
smaller than the dimensions of the device, and therefore the ED regime must be used 
to describe the field propagation. 

If T > 10Tom, then the discussion depends on the ordering between T, and Tm, Tem 
having a value in-between these two characteristic times being their geometric mean. 


5.6 Equivalent infinite circuits of devices with 
distributed parameters and their finite 
approximations 


The models with lumped parameters shown in Figure 5.11 and Figure 5.12 are valid 
not only for wire-shaped conductors, but also for massive conductors if the operat- 
ing frequency is low enough. However, with the frequency increase, the penetration 
depth of the EM field diminishes, making the field regime to move progressively to 
MQS, EMQS, and ED, eventually. This phenomenon, called “skin effect,” would make 
the lumped parameters of Figures 5.11 and 5.12 dependent on the frequency. However, 
equivalent circuits with constant lumped parameters are preferred, so that their be- 
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havior seen from the terminals is able to catch the high-frequency phenomena. This 
section shows how such an equivalent circuit can be obtained. 


The infinite electric circuit equivalent to the two-pole ECE 

Let us consider the ECE shown in Figure 5.3 with two terminals (m = 2). We will assume 
that the domain ? is linear and the charge relaxation time is much smaller than the 
magnetic field diffusion time (Te « Tm). Consequently, the appropriate field regime is 
MQS and satisfies 


ty x (iv x B00) gee (5.66) 
o H ot 


with the ECE boundary conditions (5.24), (5.25), (5.26) expressed only in E: 


ñ- (VxËC,t))=0 (V)? ezr, (5.67) 
ni-E(@,t)=0 (V)P EE-S -S, (5.68) 
xE, t =Ö0 (V)? eS US. (5.69) 


The input signal is a known voltage imposed between the two terminals: 
Ë. d? = u(t), (5.70) 
Cypcd 
where A € S, and B € S). The output signal is the current through the terminal 
i(t) = | oF - dA. (5.71) 
S, 
We will use an approach specific to functional analysis, based on the modal anal- 
ysis of this device. Let us define a linear operator A 


AÈ) = yx (5v x ËG, D). (5.72) 
o \p 


Thus the equation to be solved is 


ÈP, t) 


AE) = at 


(5.73) 


The operator A is a kind of avatar of the differential curl-curl operator, densely de- 
fined on a Hilbert space H, with compact resolvent [25]. If the domain under study is 
bounded and sufficiently smooth, the operator A with o > O and u > O is symmet- 
rical and positively defined in the space of solenoidal functions with null-type ECE 
boundary conditions. Consequently, its eigenvalues equation 


A(é;) = Apex (5.74) 
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has an infinite number of solutions O < A; < A, < --- < A, < =- > oo. These solu- 
tions are the spectrum of differential operator A, and the corresponding eigenfunc- 
tions make an orthonormal basis of H 


(Èi, é)) = 65, where (Èi, é;) = | Èi $ ê; dv. (5.75) 
D 


Thus, the solution of (5.73) for a step excitation u(t) = Uo h(t), where h(t) is the unit 
step, can be expressed as a generalized Fourier series of eigenfunctions: 


ÈC, t) = Ba + > c(t) an| Uo. (5.76) 
k=1 


This is a convergent series, with separate spatial and temporal variables. The term 
EP) is the stationary field in D, generated by the voltage u = Up = 1, with A(Ep) = 0. 
From (5.73), (5.74), and (5.76) it follows that 
dck _ 5 fap PO ay es 
Aye, (t) + ar =0 > c(t) = -be with Tk = 1/Àx. (5.77) 


Consequently, (5.76) becomes 


EGE, O = | Eo(?) - ¥ bye! èf) |Uo. (5.78) 
k=1 


The initial null condition Ë, 0) = 6 imposes that 
Èo) = X bê). (5.79) 
k=1 


This means that b, are the Fourier coefficients of the solution for the stationary regime. 
Finally, the ECE current is derived from (5.71), (5.78), and (5.79): 


i(t) = È b(1 - e*™) | oè?) - il Uo. (5.80) 
k=1 S, 
If we denote 
Ree (by | PRG da) a ER (65.81) 
Sy 


then (5.80) can be written as 


i(t) = Sy Rl — e Pl), (5.82) 
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which represents the current of a ladder circuit with an infinite number of steps con- 
sisting of RL series circuits with the parameters given by (5.81). 

Due to the linearity of the analyzed system, the obtained result is valid with null 
initial conditions for any excitation, not only for the step voltage. The infinite-ladder 
circuit (Figure 5.14 (left)) is therefore equivalent to the distributed parameter element 
D with MQS field and ECE boundary conditions, regardless of its excitation. 


The finite approximation of infinite equivalent circuits 

The modal analysis explained above is based on the computation of the problem’s 
eigenvalues and eigenfunctions. The spectrum of the operator A, which in theory can 
by continuous, is in reality a discrete one, being a numerable infinite set. Since the 
series (5.78) and (5.79) are convergent, the partial sums converge to the infinite series, 
the higher-order terms are less and less important. 

The complexity of this system can be reduced by truncating the ladder to a lim- 
ited number of steps, thus ignoring the irrelevant effect of the upper steps. This gener- 
ates an error in the stationary regime, which can be eliminated by adding a resistance 
which, paralleled with those of the inferior steps, produces the stationary regime re- 
sistance Ro (Figure 5.14 (right)): Ryyy = 1/(1/Ro — Vx I/R. 


Figure 5.14: Infinite ECE equivalent circuit (left) and its corrected truncation (right). 


Thus a sequence of finite circuits is obtained, which converges towards the exact infi- 
nite model, all having a correct stationary behavior. Although simple, this reduction 
method based on truncation is not optimal. The applied compensation corrects the 
value of the admittance in the stationary regime, but it affects the limit to infinity, 
which becomes 1/R,,,; instead of the correct null value which occurs in the absence of 
this modification. Since Rẹ increases to oo, the error goes to zero when k > oo. 

For the particular case when the element of the ECE circuit is made of a homoge- 
neous material and has a cylindrical shape of arbitrary cross-section, the eigenvalues 
and eigenfunctions are precisely those of the two-dimensional Laplace operator in the 
cross-section with null Dirichlet boundary values. The truncation errors have been es- 
timated for two special cases of practical significance, for which the analytic solutions 
are known - the plate (Figure 5.15) and the circular cylinder. 
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Figure 5.15: Dipolar ECE. 


The expressions obtained for the lumped parameters of the infinite equivalent circuit 
for the plate are 


Ry = L/(2cal), (5.83) 
Ry = RoBy/2, where B, = (2k-1)a/2, k=1,2,..., (5.84) 
Ly =paLl/(4), k=1,2,.... (5.85) 


The inductances L, have a constant value, while the resistances R, increase approxi- 
mately proportional with the square of k. That means a truncation error of about 1%, 
if only 10 steps are retained in the finite ladder approximation. 

The admittance of the plate Y(s) = tanh(VsT)/(Ro VST), with T = pod’, is a tran- 
scendental function versus s, with an infinite number of poles są = —R,/L,;. In the 
MQS regime, the poles and zeros of the admittance (impedance) are real and nega- 
tive, in a series which tends to infinity. Thus, the system is passive and stable. Fig- 
ure 5.16 represents with dots the Nyquist diagram of a plate admittance Y(jw)Ro, for 
wt =107,10°*,...,10°. 

The result of the presented approach is an analytical solution; therefore it is an 
illustration of the AAM. The ladder truncation is not the best method to reduce the 
infinite circuit to a finite one. In [32] another procedure is proposed, able to identify 
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reduced-order models for various orders, which ensures an error of minimum Hankel 
norm. It is determined that the errors of this optimal procedure are about 10 times 
smaller than those of models of the same order obtained through simple truncation 
and compensation. Hence, the circuits from Figures 5.11 and 5.12 can model also the 
skin effects in conductors, if ladder circuits with four steps, three RL series, and onea 
simple R substitute initial RL series groups. 


5.7 FEMs and their reduction 


Analytical approaches are not appropriate for EM devices with complex geometries en- 
countered in practice, in which cases numerical methods have to be used. The FEM is 
the most popular one, due to its ability to adapt for the solving of intricate geometries 
and various types of PDE that correspond to any EM field regime, or to other multi- 
physics fields. 

The theoretical foundation of FEM is the reformulation of the problem in weak 
form. In this section we will show how the EM equations for various field regimes 
can be reformulated in weak form. Then, we will review how a corresponding discrete 
model can be obtained and how its order can be reduced. 


The weak form of the fundamental electrostatic problem 

The ES potential V defined by (5.45) satisfies the generalized Poisson equation (5.46) in 
a linear domain D, with € > 0, to which boundary conditions (5.49)-(5.50) are added, 
to ensure the potential’s uniqueness. Relations (5.46), (5.49), and (5.50) represent the 
strong form of the fundamental ES problem and V is its strong solution. The weak form 
is obtained by projecting (5.46) on a set of independent directions of a Hilbert space 
H of “test” functions u: 


5 | V - (eVv)udx = | puax, (5.86) 
D D 


where the weak solution is denoted by v and p, was renamed p. By applying the Gauss- 
Ostrogradsky formula? it follows from (5.86) that 


faw - (Vu) dx = | puax + | cus dA. (5.87) 
D D aD 
This form of equation is also called Galerkin projection, and the procedure for obtain- 
ing it method of moments, since the moments are defined by the products between the 
residual and test functions, that have to be null. 


2 V-((eVv)u) = V-(eVv)ut+e(Vv)-(Vu), then apply i= and use Gauss—Ostrogradsky: fo V-((eVv)u) dx = 
fap ((EVv)u) -ñ dA. 
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By defining a bilinear functional (form) a(-,-) and a linear functional f(-), relation 
(5.87) imposes the equality between these two functionals for any test function u hav- 
ing zero values on the Dirichlet boundary: 


a(u,v) =f(u) WueH, (5.88) 
where 


a(u, v é(Vv) - (Vu) dx, (5.89) 


f(u pudx + | eufy dA, u(Sp) = 0. (5.90) 


RI 

D 
)= | 

D Sy 
The test functions u satisfy a null Dirichlet condition on Sp, while the solution v sat- 
isfies a nonnull condition on Sp. Therefore, in there are two subspaces: one of test 
functions Ho with null Dirichlet conditions, which is a linear space, and the other one 
of trial functions Hp, in which the solution is searched for; Hp is an affine (and not a 
linear) subspace of H containing the functions that satisfy the nonnull Dirichlet con- 
dition fp in (5.49). For a correct definition of the bilinear functional a(-,-), the space 
H will be considered a Sobolev space of square Lebesgue integrable functions on D: 
L°(D), which has generalized, square integrable derivatives (gradient), isomorphic 
with W°, the domain of the grad operator. The inner product in H, denoted by (u, Vays 
defines the norm in that space: 


H =H! = fu € LD) | Vu € (LAD) }; 
(U,V) 34 = | wax + [vw (vvd > Wiles = yv vn (5.91) 


D D 


The weak form of the ES problem is (5.88). It addresses the two boundary condi- 
tions Dirichlet and Neumann in a very different manner. The Dirichlet condition is 
“strongly” imposed from the start, such that the solution will be searched for in a set 
of functions, which satisfy this condition. On the other hand, the Neumann bound- 
ary condition, which is explicitly included in the expression of the linear functional 
(5.90), is “weakly” satisfied as well as possible. For this reason we say that the Dirichlet 
condition (strongly accomplished) is essential, while the Neumann condition (weakly 
accomplished) is natural. In the particular case of null Dirichlet conditions (fp = 0), 
the two spaces, that of possible solutions (trial function space, named Ansatz in 
German publications) and that of test functions 


H} = fu € LD) | Vu € (L°(D))’,u(Sp) = 0} cH}, (5.92) 
H} = {u € LD) | Vu e (L°(D))’,u(Sp) = fo} c H’, (5.93) 


are identical, Hp = Ho. By comparing the two forms, strong and weak, for the ES 
equations, we observe the following advantages of the weak form: 
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— Inthe strong form there are second-order differentials, while in the weak form the 
differentials are only of the first order, which impose fewer restrictions to func- 
tions. 

-— Inthe strong form, the boundary conditions are separately written, while in the 
weak form they are included in the equation, in the definition of the functionals, 
explicitly for the natural condition and implicitly for the essential one. 

— The strong equation is verified for each point in the computing domain, while in 
the weak form the equation is globally verified (for each basis test function). 

— Inthe weak form, the interface conditions on discontinuity surfaces are automat- 
ically fulfilled and they do not need to be explicitly imposed, which is the case for 
the strong form of the equations. 

— The material parameter may have an arbitrary spatial change in the weak format, 
since it does not need to be derivable. 


These advantages make the use of functional analysis instead of classical calculus 
worthwhile. 


Well-posed fundamental electrostatic problems 

For the functional framework given by (5.88)—(5.93), the conditions required by the 
Lax—Milgram theorem (discussed in Chapter 1 of [4]) are satisfied [8]. This theorem 
states that if the bilinear functional a(u, v) defined on the Hilbert space H is bounded, 
i. e., a positive constant C exists so that 


|a(u, v)| < Cllullilv |, (5.94) 
and coercive, i. e., it a real positive constant c exists so that 
|a(u, u)| > cull? (5.95) 


for any elements u and vin H, then equation a(u, v) = f (u) has a solution in 1 for any 
f from H’ (dual space of H, defined as the space of linear functionals over H), which 
is unique and bounded, i.e., ||v|| < ||fl|/c, where c is the coercivity constant. 

This theorem has a crucial role, since it guarantees proper expression of the ES 
problem in its weak form, providing the existence, uniqueness, and well-conditioning 
with the Lipschitz constant L = c of the solution. Consequently, this result concludes 
the mathematical modeling of ES problems. 

Since the bilinear functional a(., -) is symmetrical and positive, the solution of the 
weak form minimizes the energy convex functional F(v) given by (5.96), a statement 
known as the Dirichlet principle: 


F(v) = sal, v)-fwv) > dF= 5(a(dv,v) +a(v,dv)) - f (dv), (5.96) 
minF(v) «dF=0 © a(u,v)=f(u)y, VWu=dveH. (5.97) 


5 Complexity reduction of electromagnetic systems =——— 177 


This is the proof that, in the case of elliptical div-grad PDE equations (e. g., Laplace and 
Poisson equations), the Galerkin projective formulation (5.88) and Ritz minimizing 
formulation (5.96) are equivalent. For this reason, the solution of the weak form is 
also called the variational approach. 


Finite element method principles 

The central concept of the FEM is the use of the weak form given by (5.88) for a finite- 
dimensional subspace of H, denoted as H,, suitable for a computing system. The goal 
is to find a numeric, discrete solution v, characterized by a finite number of degrees 
of freedom. The numerical solution satisfies 


a(Uh Vh) = f(up), Vun E Hp CH, (5.98) 


where a(-,-) and f(-) are defined by (5.89) and (5.90). Therefore, the conditions in the 
Lax—Milgram theorem are still fulfilled, so the approximate discrete problem (5.98) is 
also well-posed, having a solution that is unique and is continuously dependent on 
data. This is yet another advantage of the weak reframing of the field problems. 

The finite-dimensional space Hp is generated by a finite set of basis functions also 
named shape functions or test functions; Hp consists of the set of linear combinations 
of these basis functions, so to identify the numerical solution in the case of the null 
Dirichlet boundary conditions, it is enough to find a finite number of degrees of free- 
dom, coordinates of the solution in this space Hp. In FEM, the basis functions are de- 
fined starting with a disjoint (nonoverlapping) partitioning of the computing domain 
in a mesh of cells having simple geometrical shapes (the most frequent are triangles 
or quadrilaterals in two dimensions and tetrahedrons or hexahedrals in three dimen- 
sions), called finite elements, with maximum size h. The basis functions are complete 
polynomials of degree p, i. e., they contain all terms x'y/(z*) with respect to the Carte- 
sian coordinates x and y (and z in three dimensions, compared to two dimensions, 
where k = 0), with degreei+j+k < p + 1. In each cell, the polynomial interpolates 
the values of the potential in a minimum number of nodes, points located on the el- 
ement boundary. This choice ensures the continuity of the potential at the transition 
through interfaces between elements, this being an essential FEM condition, called 
inter-element compatibility. 

The floating nodes are located in the interior of the computing domain and on the 
Neumann boundary of the domain, while those located on the Dirichlet boundary are 
not floating, since they have an imposed, fixed known potential. The degrees of free- 
dom resulting from solving a system of linear algebraic equations are those potential 
values in the floating nodes which ensure the best global approximation of the exact 
solution. The number of floating nodes N is the number of degrees of freedom of the 
problem, as each floating node defines a basis function which is a polynomial of de- 
gree p for every element, having a unitary value for that node and zero value for all 
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Figure 5.17: Basis function in linear two- 
dimensional FEM. 


Figure 5.18: Example of a three-dimensional mesh (Comsol). 


other floating nodes. For the simplest case, when p = 1, called linear (first-order) el- 
ements, the nodes are located in the vertices of the triangles/tetrahedrons. The basis 
function p, associated with the node k is null over the entire computing domain ex- 
cept for the elements which have that node in common. For the two-dimensional case 
with p = 1, the k basis function has a pyramidal graph with unity height for node k 
and the base formed by the assembly of triangles which have this node in common 
(Figure 5.17). 

The maximum size for all edges, denoted by h, gives the mesh norm, and de- 
scribes the level of mesh refining. The degree p of basis polynomials gives the finite 
element order. The most frequent orders are linear (p = 1), quadratic (p = 2), and 
cubic (p = 3). We say that the shape functions constitute a nodal base if the de- 
grees of freedom are the values of the numerical solution in the nodes, i. e., the ba- 
sis functions have the values p, € Hp P(n) = ṣi, j = 1,...,N, in triangulation 
nodes n4, nz, ..., Ny. For each cell, the number of polynomial coefficients is equal to 
the number of nodes. For example, in two dimensions, the linear triangular finite ele- 
ments have three nodes placed in vertices, and the quadratic elements have six nodes, 
three nodes being added in the middle of the edges. The coefficients are associated 
to (1,x,y) in the linear two-dimensional case and to (1,x,y, x’, y”, xy) in the quadratic 
two-dimensional case. In three dimensions (Figure 5.18), the linear finite elements 
have four nodes placed in tetrahedron vertices, the polynomial being a linear com- 
bination of (1, x, y, z). The quadratic tetrahedral elements have 10 nodes, three nodes 
in vertices, and six in the middle of the edges, the polynomial being generated by 
(1,X,V,2, X?, Y°, Z°, yz, XZ, xy). In practice quadrilateral (hexahedral) meshes are used, 
having only nodes on the boundaries of elements, internal nodes being removed, to- 
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Figure 5.19: Typical elements and their nodes. 


gether with highest-order terms of the shape functions. They are called “serendipity 
elements” and have only eight (20) nodes (Figure 5.19). 

In FEM a frequently used concept is that of barycentric coordinates, which in tri- 
angles and tetrahedrons give indication about the distances to the vertices. In the 
case of a triangle, there are three barycentric coordinates: À;, associated with vertices 
i = 1,2,3, which have affine variation inside the triangle. The coordinate A; has unitary 
value in the vertex i, and it is zero in the other two nodes. They identify the position of 
any point inside the triangle, although actually only two of them is enough, since the 
sum ofall barycentric coordinates is unitary (A, + A) + A; = 1). In the same manner the 
four barycentric coordinates of a tetrahedron are defined. The nodal shape function 
of first order associated with the node i has in each adjacent triangle/tetrahedron just 
the value Q; = A;. With these selections, any element of the solutions’ space with null 
Dirichlet conditions has the following expression: 


N 
Up = Suro; Vun € Hp C Ho- (5.99) 
jal 


By replacing (5.99) in (5.98), a system of N linear algebraic equations is obtained: 
Au =b, (5.100) 
where 
A= [alp p]; b= [fp]; u= [uj]. (5.101) 


The solution of this system is the vector of potential values in the floating nodes. The 
matrix A, called the rigidity (stiffness) matrix, and the right-hand side term f have the 
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following elements: 


ay = alp, pj) = | evp) Ve) dx 
D 


by = FY) = | ex: dx + | ef dA. (5.102) 
D Sy 


The matrix A is symmetrical, positive definite, and sparse due to the limited support of 
the basis functions. The integrals in (5.102) are estimated analytically (Holland-Bell) 
or, more frequently, numerically with Gaussian quadrature [58]. Since this quadrature 
is exact for low-degree polynomials, it gives exact values for the integrals (5.102) if 
appropriate quadrature nodes are used. 

Considering the properties of matrix A, the system of linear equations (5.100) can 
be solved with direct methods (such as Cholesky factorization type LLT, LDLT, or SVD) 
or iterative methods (such as conjugate gradient, without or with various precondi- 
tioning). Regardless of the solving method, the condition number of the rigidity ma- 
trix dictates the accuracy of the solution and the efficiency of computations. There 
are many preconditioning techniques, which avoid the ill-conditioning. In particular, 
multimesh preconditioning (called also multigrid) is an extremely efficient one, giving 
a strong acceleration of the iterative methods [10]. The essential part of the rigidity ma- 
trix A can be identified by truncating SVD factorization of that matrix. It is an efficient 
technique for the order reduction of the FEM model (see also Chapter 2 of [4]). 

The linear system (5.100) is just the nodal equation of a capacitive circuit, con- 
nected according to the FEM mesh. In the EC case, the equivalent circuit contains re- 
sistances. These circuits may be reduced by star-polygon transforms which remove the 
internal nodes. This process corresponds to the Gaussian elimination applied to the 
internal nodes in (5.100). The resulting circuit has fewer nodes, but usually, the num- 
ber of branches is increased, because the matrix is no longer a sparse one. To further 
reduce the resulted circuit, several techniques to sparsify the matrix can be applied, 
by clustering the nodes which have equal or almost equal potentials. 

The nodes on the Dirichlet boundary have known potentials, which in the general 
case may be nonnull, and need to be carefully addressed in the numerical approach, 
i. e., to preserve the symmetry of the matrix. If we separate the nodes on the Dirichlet 
boundary, we obtain for floating nodes a system of linear equations with a form similar 
to that of the problem with null Dirichlet conditions: 


N 
Un= > forPe+ YU; Vee Hn C Hs fox = fp(Pr)»Px € Sp (5.103) 
keSp j=1 


$ Soe Pi %) (5.104) 


keSp 


N 
aun, Pi) =f) = od sone.) = f (pj) - a( 
jal 
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Au = b, (5.105) 
A = [a(Q;, p;)], b=([f(@) + fp], u= [u;'], (5.106) 
aj = alpi, pj) = [evo - (Vep;) dx, (5.107) 
D 
b; = f (p) + fni = | pp; ax + | afro, dA- > arfor (5.108) 
D Sy keSp 


This time, the right-hand side term has three contributions: internal field sources 
(charge density inside the cells); boundary natural conditions (type Neumann, ad- 
dressed similarly with a superficial charge density); and the boundary essential con- 
dition (type Dirichlet, addressed similarly with the floating nodes potentials, but 
imposed forcefully). In this manner, the numerical solution will satisfy exactly the 
essential Dirichlet boundary conditions, while the Neumann, natural boundary con- 
ditions are satisfied as exactly as possible. Consequently, these results conclude the 
step of numerical modeling. 


FEM, from theory to code 

If we suppose that the problem was previously detailed and well formulated, know- 

ing the computational domain, the solved equation, values of the material constant, 

and the field sources, including the boundary conditions, the general FEM has the 
following steps of the computational modeling stage: 

1. Domain discretization selects the shape of the finite elements and generates the 
mesh which covers the computational domain, determining the mesh norm h. 

2. Selection of the element type consists of choosing the shape functions, in particu- 
lar the elements order p and node placement. 

3. The weak formulation step identifies the bilinear and linear functionals of the 
weak form, considering the equation to be solved and the boundary condition. 

4. The derivation of element matrices step identifies the coordinate transformation 
which maps the local coordinates of a common reference element to global ones, 
and thus its Jacobian matrix, and selects the variant of the Gaussian quadrature 
method used to compute element contributions to both functionals. 

5. Assemblage of element equations computes and localizes the contribution of each 
element to the system matrix and to the right-hand side term, evaluating both 
bilinear and linear functionals, respectively, treating with a different technique in 
particular the Dirichlet boundary conditions. The use of sparse matrix techniques 
is highly recommended since it produces superior performances in execution. In 
most software packages, steps 4 and 5 are encapsulated, being hidden from the 
user. 

6. Solution of equations is a step in which the linear system generated previously is 
solved with a direct or an iterative method. As will be seen, in the case of transient 
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problems, the discretized equations to be solved are of ODE type, which require 
a numerical integration in time. A different approach is used to find the eigen- 
values in the case of modal analysis, when the problem is reduced to finding the 
eigenvectors and eigenvalues of a matrix. 

7. Order reduction may be realized by iterative cycling of this procedure and/or by 
processing simulation results or the state equations generated with the FEM. 


Examples of simple FEM codes illustrating these steps can be found in [20]. 

In FEM programs, most often, coefficients (5.102) are not computed directly, but 
through a bijective transformation from triangular, local coordinates to physical, 
global coordinates. This transformation maps a standard reference element (e.g., a 
right angle triangle with unity legs, a unitary square, a unitary tetrahedron with or- 
thogonal, unity legs, or a unitary cube, with natural, local coordinates), to elements 
in the physical space (called global coordinates), being determined by the correspon- 
dence of the nodes in these two spaces. These elements are called isoparametric. As 
the main advantage, this approach allows the encapsulation and separate develop- 
ment of the reference element library. When implemented on a computer, a difference 
is acknowledged between the basis functions, defined on the reference cell, and the 
shape functions, defined in the real physical space, such that each shape function is 
associated to a degree of freedom from the global grid. 

The derivation of element matrices and the assemblage of element equations are 
the core steps of any FEM program. The combination of two ideas, isoparametric ele- 
ments and Gaussian numerical quadrature, allows a simple and uniform treatment of 
the elemental assembly procedure, even for higher-order elements and elements with 
curved boundaries [18]. 


Convergence of the finite element method 

The convergence of the FEM guarantees that the numerical solutions will tend toward 
the exact one when the discretization mesh is uniformly refined and thus the number 
of degrees of freedom tends to infinity. 

The convergence towards the exact solution assumes that the discrete, finite- 
dimensional space H,, tends to an infinite-dimensional space, dense in H. This con- 
vergence condition, named completeness, is obtained if the shape functions are con- 
tinuous (ensuring the inter-element compatibility) and they have the unity partition 
property. Under the conditions of the Lax—-Milgram theorem, the completeness guar- 
antees the convergence [18]. 

A theorem which provides the rate of convergence of the numerical solution v} in 
FEM assumes that the shape functions are polynomials of p degree over each element 
of a mesh having the norm h, satisfying the inequality [55] 


Iv- vpli < CHO Wap (5.109) 
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if the exact solution v, continuous on the entire domain, is sufficiently smooth. 

This relation known as a priori error estimators guarantees the global convergence 
of the FEM, showing that if the norm h of the discretization mesh tends to zero (which 
implies that the number of degrees of freedom N tends to infinity), then the numerical 
solution v, tends to the exact one v. We observe that (5.109) ensures a global, “average” 
convergence of the numerical solution and its m derivatives towards the exact solution 
on the entire domain, as opposed to a local, pointwise convergence, characteristic of 
strong formulations of problems. If m = 0, it refers to potential, and if m = 1, it refers 
to the field strength. 

The numerical solution is trusted if it is relatively independent of the mesh, i.e., 
h is sufficiently low, so that the error (5.109) has an acceptable level. The mesh in- 
dependence may be checked by refining the current mesh, e. g., by halving h. If the 
numerical solution does not change too much, the mesh is adequate; otherwise the 
refining process continues. This approach is named uniform mesh (or “h”) refining. If 
new nodes are added in the middle of each edge, face, or cell, the number of degrees of 
freedom increases just as when passing from the first-order to the second-order FEM. 
Although the rigidity matrices have the same size, in the first-order refined case it has 
fewer nonzero elements. The additional computational effort to generate and solve 
the system in the case of the quadratic FEM order is rewarded by a more accurate and 
smoother numerical solution, since the order of convergence for the field is just the 
order p of the elements. 

Figure 5.20 (left) illustrates the convergence of the FEM in the first- and second- 
order cases. The faster convergence of the second-order elements (p = 2), related to 
that of the first order (p = 1), is obvious. According to this figure, we observe that in 
practice, the numerical error cannot be made lower than a limit, which is about 1078 to 
1070, If the individual mesh elements start to get very small, we run into the limits of 
numerical precision. That is, the numbers in our model are smaller than can be accu- 
rately represented on a computer. This is an inherent problem with all computational 
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Figure 5.20: Left: Relative error vs. number of degrees of freedom (https://www.comsol.com/). 
Right: Relative AMR error vs. number of nodes [6]. 
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methods, not just the FEM; computers cannot represent all real numbers accurately. 
Moreover, the condition number of the FEM matrix grows with mesh refinement at 
least linearly (or faster, depending on the problem). Therefore, it is important for the 
users to be aware of this rapidly growing accumulated round-off error as they refine 
meshes, especially with very large models. One million degrees of freedom in three 
dimensions is a usual limit for mesh refinement. 


Adaptive mesh refinement 
The nonsmooth problems have a lower convergence rate. One method to recover the 
depreciation of convergence caused by singularities is to use a self-adaptive algorithm 
to refine the discretization mesh. This technique is called adaptive mesh refinement 
(AMR) or the adaptive FEM [6]. This iterative algorithm begins from a coarse mesh 
and solves the problem on a series of meshes, adding successively new nodes, only 
when and where necessary. Consequently, the number of elements and the number 
N of degrees of freedom are progressively increased, but not so fast as in the uniform 
refinement. At each iteration, certain elements, specifically those for which the solu- 
tion has an unacceptable error, are refined through segmentation. For example, in two 
dimensions, a triangle can be divided in two, three, or four subtriangles, depending 
on how supplemental nodes (zero or one) are appended in the middle of each edge. 
Figure 5.20 (right) shows the mode in which the relative error decreases with re- 
spect to the number of nodes N (equal to the number of degrees of freedom), in the 
case of a uniform refinement of the mesh and in the case of an adaptive refinement. 
In the first case the convergence rate is approximately 0.7, two times lower than q = 
1.4 = 4/3 < 2 (which is the ideal value in the absence of the singularity). These results 
were obtained for the example shown in Figure 5.21, where the discretization mesh is 
generated by AMR. It can be seen that the mesh is selectively refined, especially near 
the vertex that generates the singularity. 


are Figure 5.21: Multimesh FEM with adaptive refine- 
ment [6]. 
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Error convergence 


Figure 5.22: FEM convergence h for p = 1 
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Finite element with hp-refinement 

As previously observed, another method for error reduction of numerical solutions is 
to increase the degree of the polynomials, which define the shape functions. The re- 
finement by successively increasing of the degree p provides very good results for the 
case of extremely smooth solutions (which are analytical functions, indefinitely dif- 
ferentiable functions, harmonic functions, solutions to the Laplace equation, etc.), for 
which the error diminishes exponentially with the increase in the degree p. However, 
it is useless in the case of nonsmooth problems. Combination of h- and p-adaptive 
refinements regains the accelerated convergence for problems with singularities. Its 
justification is based on the observation that in zones where the solution is smooth, 
it is more efficient to use high values for the degree p, while in zones adjacent to the 
singularities, the spatial refinement through reduction of the h parameter is more effi- 
cient [41]. The hp algorithms are super-convergent, ensuring an exponential decrease 
of the error, which cannot be achieved with h or p strategies alone. As can be seen 
in Figure 5.22, the L-shape problem is solved through h-refinement at a convergence 
rate of 2/3; through p-refinement at a rate of 5/4, and through hp-refinement with an 
exponential convergence, for which real numbers C, c, and 6 exist so that 


6 
lv- vall < Ce. (5.110) 


Through hp-refinement field relative errors of the order of 10~° are obtained, i.e., 
numerical solutions with six correct significant digits, with fewer than 10° degrees 
of freedom. Since uniform h-refinement requires a much larger number of degrees of 
freedom to achieve a similar accuracy, the hp-refinement may be considered as being 
the most efficient method for complexity reduction of numerical FEM solutions. 

AMR and hp-refinement are techniques which allow the reduction of the number 
of degrees of freedom, for a given accuracy. Therefore, they are improvements of the 
numerical model, which provides on-the-fly MOR, the latter being the most efficient 
known technique of this category. 
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The weak form of curl-curl equations 

Static and stationary magnetic regimes (MS, EC) are similar to the ES regime for non- 
charged bodies, and their scalar potential satisfies the same generalized Poisson div- 
grad equation, which can be solved as discussed above. The magnetostationary (MG) 
regime is an exception, which needs a magnetic vector potential, since in this case 
a native scalar potential does not exist. Essentially, a PDE of curl-curl type (5.57) is 
solved to determine the magnetic field, whereas for all other static and stationary 
regimes, a PDE equation of div-grad type (5.46) is solved. 

The weak form of MG regime equations with mixed boundary conditions is ob- 
tained by projecting the residual on the space of test functions, which are now vectors. 
This set is actually the Sobolev space of the vector square integrable functions, which 
have a curl of integrable square: 


3 


Ho(curl, D) = {ù € (LAD) |V xù e (LD), À xù = Gon Sp}. (5.111) 


In this functional framework, the variational formulation of the problem searches 
for the solution Ÿ, such that 


a(u,V) =f(@i) Yù € Ho(curl, D), (5.112) 
where 


v(V x Ÿ) - (V x ù) dx, (5.113) 
J-ùdx+ [Is -naa. (5.114) 


We will proceed as in the scalar case, where it was found that the solution v with null 
essential boundary conditions was sought in the linear space Ho, as a function with 
such essential boundary conditions. If the boundary condition was nonnull, the so- 
lution v was sought in the affine space resulting from translating Ho, with elements 
which meet that nonnull condition. This time, according to (5.114) the essential bound- 
ary condition A, = (ñxÅ)xñ is that on Sp c ƏD, while on Sy = dD-S, natural boundary 
conditions H, = (ñ x vV x A) x ñ = J, are imposed, and the vector solution must be 
sought in a curl-conform subspace, in which the vector fields conserve their tangential 
components when crossing discontinuity interfaces. 

However, regrettably in this instance of three-dimensional MG, the functional 
a(u, Vv) is not coercive, and the Lax—Milgram theorem cannot be directly applied. Con- 
sequently, the nongauged magnetic potential vector, solution of equation (5.112), is 
not unique. An exception is the two-dimensional plan-parallel case, with two mag- 
netic field components in the problem’s plane and a sole component for current, and 
vector potential (orthogonal to that plane), which is automatically gauged, having a 
null divergence. Moreover, the sole component of vector A satisfies a curl-curl-type 
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equation which devolves in one of type div-grad. To surmount this standoff, there are 
several solutions: 


Impose gauge. For the magnetic potential vector, this is most simply achieved 
with the Coulomb condition (V - A = 0) and by adding the boundary condition 
A, = 00n Sp. 

Do regularization. Another approach, called regularization, consists in adding a 
coercive term to the bilinear functional, meaning that a mass matrix is added to 
the stiffness matrix, weighted with a parameter k: 


alii) = | viv xi) V xi dx + k | V- did. (5.115) 
D D 


Due to this term, the coercivity condition is met again, and the problem is well 
formulated, no matter how small is the parameter k. Then, the solution is moved 
towards the limit for k — 0. 

Use Lagrange multipliers. In the case of the weak form of curl-curl type, another 
possible approach consists of extending the Lax-Milgram theorem such that it 
can be applied to variational problems with Lagrange multipliers, which do not 
have minimum points, but have critical points of saddle type. Such an extension 
is achieved by the Brezzi theorem, which highlights the conditions to have a well- 
posed weak formulation of the equation for vector potential [55]. 

Use reduced scalar potential. The idea behind this method is to extract a field 
with a curl from the solution, which is precisely the given current density, such 
that what is left is a nonrotating field, solution of an MS problem. Thus, the prob- 
lem accepts a scalar magnetic potential, called reduced. After its numerical deter- 
mination by solving a div-grad equation, this field is added to the field extracted 
initially, resulting in the numerical solution of the MG problem. Other expressions 
with different potentials of the magnetic field in MG and MQS regimes are pro- 
posed in [37]. 

No gauging. This is a very interesting approach in which the potential vector A 
is left without gauge and its indeterminate equation is solved numerically with 
an iterative method of Krylov type, which has native minimization properties. In 
addition to surprisingly reaching a unique magnetic field, the iterations are faster 
convergent than in the gauging case [26]. 


Analysis of the MG field with edge elements 

When computing resonance frequencies for cavities, false resonant modes, called spu- 
rious modes, emerge when nodal finite elements are used. This phenomenon, called 
spectral pollution, disappears when the vector solution is sought in a curl-conform 
space. The reason is that in H,(D) the kernel of the curl-curl operator is not properly 
represented [6]. The response is to use shape functions appropriate for the vector prob- 
lem, called edge elements [8]. In three dimensions a vector field is represented by the 
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three coupled scalar fields, components of the vector field: 
T =T,é,+T,é,+T,é,, (5.116) 


where é,, ey: and é, are the unit vectors of the Cartesian system of coordinates. In prin- 
ciple, each component T,, T,, and T, could be represented by combinations of nodal 
shape functions, but as mentioned before, this approach should be avoided. It does 
not guarantee the conservation of the normal component of the field flux density (or 
equivalently, the tangential component of the magnetic vector) either when passing 
from an element to another, or on the discontinuity surfaces. An alternative is to seek 
numerical solutions resulting from the expression 


E 
T (x,y,z) = Y twy, Zz), (5.117) 
i=1 


for which the degree of freedom t; is the integral of the vector field T along the edge 
l; from the assessed triangulation with E floating edges. Therefore, vector shape func- 
tions Ww; are defined by 


| ice y,z) ds = 5y. (5.118) 
i 


Consequently, the vector shape function wW; is tangentially oriented along the edge l; 
and is normally oriented along the other edges of element K which includes l;. For this 
reason, such basis functions are called edge elements. Furthermore, in these circum- 
stances the vector shape function w, has in three dimensions the tangential compo- 
nent null on any face which does not include the edge i. The encountered restrictions 
cause two triangles in two dimensions with a common edge to have on the interface 
through this edge the same tangential component of the shape functions, but a differ- 
ent normal component. In three dimensions the vector shape functions conserve the 
tangential component where passing from a tetrahedron to another by way of their 
common face. Asa result, the tangential component of these “edge” vector shape func- 
tions is continuous, in all computing domains, for which reason they are also called 
vector shape functions with continuous tangent. This curl-conformance makes them 
appropriate for representing the electric or magnetic field strength, but also for the 
magnetic potential vector (in which case the conservation of the normal component 
of magnetic induction is automatically ensured, where passing through any surface, 
hence the solenoidal character of this quantity). 
The vector functions in barycentric coordinates are 


valid both in two dimensions, where i,j = 1,2,3, and in three dimensions, where 
i,j = 1,2,3,4, and where ij is the edge connecting the nodes i and j. These functions, 
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called Whitney elements [55], are appropriate to be selected as shape functions for the 
magnetic potential vector, since they have a constant tangential component of 1/l;; on 
the ij edge and null on the other edges, divergence zero on the entire K element, and 
constant curl for that element: V x Ww, = 2VA, x VA;. The degrees of freedom associated 
with these functions are the integral values on characteristic edges [8]. 


FEM of MG field 
The numerical solution ¥, of the MG problem, described by the vector t = [t,], is found 
by solving the following system of linear equations: 


At =b, (5.120) 
where 
A= [aww]; b= F0]; t= [4], (5.121) 
with 
ay = [w x W,) : (V x W;) dx, 
D 
b; = [7 maxs [I maa, (5.122) 
D Su 


where v = 1/y, J is the current density, and J, is related to H, on the boundary. 

Again, the central part of the FEM algorithm consists of assembling the matrix A 
of the linear system and of the vector f of the right-hand terms. To do this, elements of 
triangulation are sequentially traversed and the contribution (5.122) of each element K 
is added to the rigidity matrix and to the right-hand side term. Then, this linear system 
is solved with direct or iterative methods and the degrees of freedom t; are determined, 
one for each floating edge. The numerical solution is the linear combination of the 
basis functions (of the edge), with these degrees of freedom as coefficients 


E 
Vp = A(x, y,Z) = $ tiw y,Z) = tw. (5.123) 
j=l 


The linear system (5.120) is similar to the mesh equations of the system of partial 


inductances, connected according the FEM mesh. Its robust reduction is discussed in 
[17]. 


FEM of the MQS field in the time domain 
The established functional framework finds its application in the MQS regime as well, 
in which there are two approaches: in the time domain and in the frequency domain. 
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The weak form of the MQS equation in the time domain with interface conditions and 
boundary conditions identical with those in the MG regime and with the initial condi- 
tion for the transient potential vector A is obtained by the Galerkin projection 


au,v)=f@i) Yù € Ho(curl, D), (5.124) 
where 
a(ii,7) = [w x?) - (V xii) dx, (5.125) 
D 
3 > ov 2 Bo 
fù)= (i.-0(S+wv)) wars [7 naa. (5.126) 
D Sy 


Actually, the difference in MQS is that, to the current density, the field source in MG, is 
added the induced current density. In additions, the electroconduction (EC) equation 
V - (aVV) = O needs to be solved for the scalar potential V, with Dirichlet boundary 
conditions (the potential value V = fp) or Neumann boundary conditions (the value 
of the normal derivative of the potential dV /dn = fy, given by the normal component 
of the current density J,, of the injected in the surface). In the weak form, the scalar V 
solution is sought in Hy given by (5.93) such that 


b(u,v)=g(u), We Ho (5.127) 


where H} is given by (5.92) and 


buv) = fow . (Vu) dx, (5.128) 
D 

g(u) = | oufy dA. (5.129) 
Sy 


The dependency between the scalar potential V and the input signals (boundary 
conditions — boundary potential and the currents injected in the boundary) is instan- 
taneous, without any delay, since V is the solution ofa scalar-elliptical problem with- 
out time variable. Using a discretization with edge elements, the vector potential is 
expressed as A = t’ W, where t is the vector of degrees of freedom and W is the vector 
of shape (edge) functions. A system of first-order ODEs is obtained by performing the 
Galerkin projection on this finite-dimensional space of test functions: 


dt 
E— = f, .130 
aq S (5.130) 
where 
S= [sy] = | v0 x tH) «(Vx iH) de, (5.131) 


D 


5 Complexity reduction of electromagnetic systems —— 191 


f= [f] = | Ji -o0VV)-w;dx + [I maa, (5.132) 
D Su 

E = [e;] = [oww - (Vii,) dx. (5.133) 
D 


Thus, if ECE type boundary conditions are used, an I/O linear time-invariant dynamic 
system is defined, which has as state vector the degrees of freedom of the edge ele- 
ments (circulations of the vector potential A along the floating edges of the discretiza- 
tion mesh). Its state matrix is actually the rigidity matrix S, E (representing damping) 
is the descriptor matrix, and the input signals are the injected currents or the applied 
potentials. Due to reciprocity and passivity, matrices S and E are symmetrical and pos- 
itive definite, and the real and negative eigenvalues A, of the matrix E~'S give the sys- 
tem’s time constants Tg = -1/A,. The system (5.130) is similar to the system of state 
equations in descriptor form of the RL circuits, which have the topology described by 
the FEM mesh. The reduction of such circuits is treated in [19]. 

Now it becomes apparent that the application of the hp-refinement, which en- 
sures a minimal number of degrees of freedom for an acceptable accuracy, is an on- 
the-fly MOR approach. However, there is a supplementary restriction: in order to en- 
sure a good accuracy, the cell size h must be smaller than the field penetration depth 
(h < 6 = /2/(wuo)). By applying a posteriori MOR classic methods (e. g., Krylov, bal- 
anced truncation, modal analysis of data, or circuit reduction) to the system (5.130), a 
model with a more reduced order is obtained eventually. A posteriori MOR is a feature 
encountered in all important FEM software packages (see Chapter 13 of this volume 
[5]), such as ANSYS or COMSOL [46]. 


FEM of the ED field in the time domain and MOR 
In the case of the ED regime, the magnetic vector potential satisfies the hyperbolic 
PDE: 


- OA OA oV\ > 
Vx (VÄ +o +e2F row +ev( 7) =}, (5.134) 


with boundary conditions (5.21) and (5.22) and initial conditions for A, 0A /ot, and V 
to have a unique solution if V- A = 0. The weak form of (5.134) is obtained by Galerkin 
projection: 


a(tuv) =f) Yù € Ho(curl, D), (5.135) 
where 


alii, ¥) = | v(V x ©) - (V xi) dx, (5.136) 
D 
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a a av V ƏV s a 
fü) -JỌ o( z t vv) e( p + v( a ))) -udx l J,- ùdA, (5.137) 


to which the weak form of the EC equation for the scalar potential V is added, as in 
the MQS case. 

By using a discretization with edge elements, the potential vector is A = t'w, 
where t is the vector of degrees of freedom and wis the vector of shape-edge functions. 
The weak form becomes the system of second-order ODEs 

dt dt 


M— +E— -St =f, 1 
pe S (5.138) 


where S is given by (5.131), E is given by (5.133), and 


> oVV\ _ zo 
f= f= [(i-ovv -eE ) tia s [J -vipaa, (5.139) 
D Su 


D 


This time, the system is characterized by three matrices — of mass M, of damping 
E, and of stiffness S. The elements of these matrices are the inner products of shape 
functions (or their derivatives) weighted with several material constants. For instance, 
in M (characteristic to the inertial phenomena in mechanical systems), the dielectric 
permittivity describes the capacitive effects; in E (characteristic to the damping phe- 
nomena), the conductivity describes the conduction effect; and in S the magnetic per- 
meability describes the inductive effects, similar to elastic stiffness/rigidity. 

The system (5.138) is similar to the equations of the RLC circuits which have the 
topology described by the FEM mesh. The reduction of such circuits is treated exten- 
sively in the literature [40, 48, 49]. Several methods for order reduction of second- 
order ODE systems of (5.138) type are also proposed in the literature [2]. The similar- 
ity to mechanical systems (see also Chapter 2 of this volume [5]), also suggested by 
the names of the three matrices, allows the use of reduction methods from structural 
analysis in electrodynamics, including the static condensation of matrices S and M, 
called Guyan reduction. This reduction consists of the selection of a reduced num- 
ber of master degrees of freedom and the removal of the other state variables from 
the stationary regime equation, in which the field sources associated with the omitted 
variables are ignored. Retaining for the dynamic regime as well only the most impor- 
tant, master variables, the state equation reduces correspondingly its dimensions. It 
is expected that the first eigenvalues will not be changed substantially and, therefore, 
its dynamic behavior will not be fundamentally affected. The hp-refinement ensures 
on-the-fly MOR. However, this time, for a good accuracy, the cells dimension needs to 
be much smaller than the wavelength of the field h « A. The numerical integration 
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over time for this system can be made also by discretizing the time derivative with fi- 
nite differences. The Courant—Friedrichs—Lewy (CFL) condition compels the time step 
to be smaller than the time necessary for the EM wave to travel through one cell. It 
ensures the numerical stability of the computation, limiting the size of the time step 
used in the numerical integration over time (At < h/c). Itis observed that, as for the ex- 
plicit scheme, the factorization of the mass matrix M becomes necessary. An approach 
which removes this effort consists of approximating the matrix M with a diagonal one, 
an operation called mass lumping, which has sufficient precision only for rectangu- 
lar cells (squares, hexahedrals), but not for simplex cells (triangles, tetrahedrons). For 
the case of triangles/tetrahedrons it is preferable to use the implicit Newmark scheme: 


=D) = 
mi 2 tte pk ia +$(6t,,, + (1—20)t, + Oty_,) 


sk f(Ot;.,, ok (1 pF: 20)t, + Otz). (5.141) 


This scheme is stable for any time step if 0 > 1/4. However, the solution loses accuracy 
if the time step exceeds the limit imposed by the CFL condition [6]. If 9 = 0, the scheme 
becomes that of explicit integration over time with finite differences. 

Both simulation and order reduction are more difficult in the ED regime than in 
the MQS and EQS regimes, since now the system eigenvalues are complex. For this 
case there is a large collection of methods for a posteriori MOR of the system gener- 
ated by FEM [11, 56, 50, 57, 45, 36, 53, 7, 1, 51]. They are based on truncated balanced 
models (see Chapter 2 of [3]), moment matching (see Chapter 3 of [3]), or data-driven 
and interpolation (see Chapter 6 of [3]). The research in the area of MOR of EM devices 
functioning with ED or MQS fields continues, since finding the best technique to solve 
this problem is still an open problem. The passivity enforcement of the reduced model 
is discussed in Chapter 5 of [3]. 


FEM of the ED field in the frequency domain and MOR 

In the time-harmonic ED regime, the complex (operational) magnetic potential satis- 
fies a complex equation of Helmholtz type, obtained after applying the Laplace trans- 
form to (5.134) with zero initial conditions: 


V x (VV x A(s)) + (Ø + €s)(sA(s) + VV(s)) = J;(s), 
V-(aVV(s)) = sp. (5.142) 
The weak forms of these equation are obtained by Galerkin projection: 
a(u,v) =f(ù) Yù € Ho(curl, D), (5.143) 
b(u,v)=g(u) Yu € Hį(D), (5.144) 
where 


aū, ?) = [vvxm -(xidx+s [0+ e5)0-tdx, (5.145) 
D D 
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= 


f(a) = ja - (0 + £s)VV) -Ù dx + | J,- ùdA, (5.146) 


D Su 


b(u, v) is given by (5.128), and 


g(u) = | spudx + | oufy dA. (5.147) 
D Sw 


Here the vector potential A is labeled as 7 and the scalar potential V is labeled 
as v. In the case of the ECE, with null initial conditions, Sz means A, = 0 on the entire 
boundary, whereas Sp means that the scalar potential is given on the terminals. In this 
case 


Fares fo +€s)(VV) -itdx, (5.148) 
D 
g(u) = | oufy dA = 0. (5.149) 
Sy 


We need to note that, as opposed to the case of elliptical equations, for which the 
Galerkin projection formulation is equivalent to the minimizing Ritz formulation, in 
the case of hyperbolic equations this equivalence disappears. Now the solution in the 
variational formulation corresponds to a critical point which, however, is not anymore 
a minimum of the energy functional, since the energy functional is not a convex one, 
having a saddle shape in the critical point, of the solution. 

As for the MQS case, the numerical ED solution is sought with nodal elements for 
the scalar electric potential and with edge elements for the magnetic vector potential, 
by solving the following systems of linear complex equations: 


aùn Yn) = fÙ) Vip, € Ho, (curl, D), (5.150) 
b(Up Vn) =8(Up) Yup € Ho, (PD), (5.151) 


where f and g are linear complex functionals, which generate the right-hand side term 
of the system of equations, while a and b are bilinear functionals (a is complex and 
bis real), which generate after the discretization the system matrix. It is the complex 
transform of (5.138): 


(s°M+sE-S)t(s)=f(s) = t(s), (5.152) 
A(x, y,Z,8) = t'(s)Wix,y, z). (5.153) 


The total number of degrees of freedom is equal to the number of floating edges, but 
before solving a linear system with size equal to the number of floating nodes, to find 
the scalar potential. These complex systems may be solved with direct or iterative 
methods. Unfortunately, this time, the multimesh approach is not applicable because 
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it is not convergent, but the methods of Krylov type are efficient, especially with pre- 
conditioning. 

The solution of system (5.152) depends linearly on the right-hand side term and, 
therefore, in the case of the ECE, with null initial conditions, in the absence of internal 
sources, the solution vector and, implicitly, the output signals depend linearly on the 
excitations of the terminals. The linearity allows the correct definition of the admit- 
tance Y and, implicitly, computation of the scattering, complex operational matrix S. 
Again, in this instance the numerical solution to the problem fora series of frequencies 
allows the determination of the frequency characteristics of the EM systems Y (jw). On 
this base, applying MOR techniques based on data, such as vector fitting (see Chap- 
ter 8 of [3]) very efficiently reduced models of these systems can be extracted [13]. 


FEM of the ED field in the frequency domain with ECE boundary conditions 

The previous formulation used the pair (A, V) in every point of the computational 
domain. Alternatively, a formulation in (E , V) can be used, as described in [21], where 
V is defined also inside the domain, or a formulation in (E , V), where the electric field 
E is defined strictly inside the domain and the electric scalar potential V is described 
solely on dD. The following weak equation for E is obtained [15]: 


fiov x Ë) - (V x Ē') + jw(o + jwe)Ë - E'] dx =jw Y Vile (5.154) 
D keZ, 


where Z, is the set of indices of current excited terminals. The equation for the electric 


potential on the boundary is obtained by projecting the normal component of the total 


current density J, mot (V x H) - n onto a set of scalar test functions V’: 


m 
(0 x H) AV" ds = P Jv’ ds = > | nv’ as = AN 
oD oD 


k=1$, keTe 
where Ë ¢ Hp, V € Hy, Ë' € Hgo, V' € Hyo, and 
He = fu c€ H(curl, Q) | n x (u x n) = -V,V' on ðQ, V' € Hy, 
m 
n x (u x n) = oon| Jsi}, 
k=1 
Heo = fu € H(curl, Q) | n x (u x n) = -V,V' on dQ, V' € Hyo 


m 
n x (u x n) = oon sil, 
k=1 


Hy = {u € H(grad, ðQ) | u = V; on Sp, k € T, 


u = constant(unkown) on Sp, k € Te}, 
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Hyo = {u € H(grad, dQ) | u = Oon Sp k € T, 


u = constant(unkown) on Sp, k € Ze}. 


The model complexity is reduced since the degrees of freedom are scalar quantities 
associated to edges strictly inside the domain and nodes on the boundary. 


5.8 Conclusions 


The optimal design of EM field devices requires the solving of Maxwell PDEs. Conse- 
quently, the state of such a device is completely described by local and instantaneous 
vector quantities, i. e., by a system with an infinite number of degrees of freedom de- 
pending on both space and time. Classical MOR approaches can be applied only to 
finite-dimensional systems, and that is why complexity reduction of Maxwell-based 
models is needed. Complexity reduction is obtained by discretization of field quan- 
tities and conduces to a model described by a finite number of degrees of freedom, 
expressed as a system of ODEs. The accuracy and order of the reduced EM model de- 
pend not only on the chosen a posteriori MOR method, but also on the a priori and 
on-the-fly complexity reduction approaches. A posteriori reduction, i.e., the reduc- 
tion after the discretization, can be carried out with one or several methods described 
in [3, 4], but the complexity reduction can be carried out only with methods specific 
to electromagnetism, which were described in this chapter: ECE boundary conditions, 
an appropriate EM field regime, geometrical simplification, and discretization meth- 
ods and meshes adapted to the problem. This chapter described FEM discretization, 
which is the most popular, but other approaches are also used in practice, such as 
the boundary element method and the finite integration technique/finite difference 
method, which generate systems of ODEs that can be reduced with the same methods 
as the ones used for the FEM. 
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6 Model reduction in computational 
aerodynamics 


Abstract: Computational aerodynamics has become an indispensable tool in the de- 
sign and analysis of modern aircraft. However, traditional high-fidelity aerodynamics 
simulations can be computationally too expensive for scenarios that require responses 
in real time (e. g., flow control) and/or predictions for many different configurations 
(e. g., design-space exploration and flight-parameter sweep). The goal of model re- 
duction is to accelerate the solution of unsteady and/or parameterized aerodynamics 
problems in real-time and/or many-query scenarios. In this chapter, we survey model 
reduction techniques for linearized and nonlinear aerodynamics problems that have 
been developed in the past two decades. We discuss essential ingredients of model 
reduction: stable and efficient projection methods, generation of the reduced basis 
tailored for the specific solution manifold, and offline-online computational decom- 
position. We focus on techniques that are designed to address challenges in aerody- 
namics — nonlinearity, limited stability, limited regularity, and wide range of scales — 
and have been demonstrated for multidimensional aerodynamic flows. We highlight 
successful applications of model reduction for large-scale aerodynamics problems. 


Keywords: aerodynamics, model reduction, parameterized partial differential equa- 
tions, (Petrov—)Galerkin projection, reduced basis 
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6.1 Introduction 


6.1.1 Motivation 


With advances in both computational algorithms and hardware, computational fluid 
dynamics (CFD) has become an indispensable tool in the analysis and design of 
aerospace vehicles. Today’s CFD tools can accurately predict aerodynamics of aircraft 
in cruise conditions and complement wind-tunnel and flight tests in the aircraft de- 
sign process; in fact, with the advances in CFD, the number of wings tested in the 
design of a typical commercial aircraft has decreased by an order of magnitude from 
the late 1970s to the early 2000s [38]. 

However, there are computational challenges that still remain out of reach for tra- 
ditional CFD solvers. To motivate the model reduction work reviewed in this chapter, 
we name a few “grand challenges” outlined in vision papers [51, 61]. First is high- 
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fidelity aerodynamic database generation; the task requires accurate prediction of 
aerodynamic forces for the entire range of flight conditions with variations in, e. g., 
the free stream Mach number and angle of attack. Second is real-time dynamic flight 
simulation; the task requires aerodynamic or aeroelastic simulation of maneuvering 
aircraft with the control input specified in real-time. Third is probabilistic design of 
cooled turbine blades; the task requires accurate characterization of the turbine blade 
performance under geometric uncertainties due to manufacturing variabilities. These 
tasks are challenging for traditional CFD solvers because they require (i) predictions 
for a large number of configurations (i.e., many-query) and/or (ii) real-time predic- 
tions of transient phenomena. Completing these tasks, especially in the time scale and 
computational resources available in typical engineering settings, can be prohibitive 
with traditional CFD tools. The objective of this chapter is to survey the state of the art 
in model reduction for many-query and/or real-time problems in aerodynamics. 


6.1.2 Real-time and many-query scenarios 


We now provide examples of many-query and/or real-time engineering scenarios to 

which model reduction has been applied. We restrict ourselves to problems in aerody- 

namics, rather than more general fluid dynamics; we refer to Chapter 9 of this volume 
for the latter. We do not attempt to provide a comprehensive review; we merely present 

a few representative works. 

S1. Aerodynamic shape optimization. One of the many-query applications of model 
reduction in aerodynamics is shape optimization. Reduced-order models (ROMs) 
are used to accelerate aerodynamics analysis under parametric geometry changes 
and to optimize the geometry. The task consists of three steps: parameterization of 
the geometry; construction of a ROM; and identification of the optimal geometry. 
ROMs have been used in many-query analysis [6, 69] and inverse design, where 
the objective is to identify airfoil geometry that yields the prescribed pressure dis- 
tribution [43, 44, 45, 78]. 

S2. Flight-parameter sweep. Another many-query application of parametric model re- 
duction in aerodynamics is flight-parameter sweep. ROMs are used to accelerate 
the prediction of aerodynamic forces and moments for a range of flight conditions 
described in terms of the angle of attack and Mach number [80, 79, 66, 68, 75, 76]. 

S3. Aeroelasticity. One of the classical real-time applications of model reduction in 
aerodynamics is aeroelasticity. The goal is to analyze the interaction between 
aerodynamics forces and elastic structure and to detect, for instance, the onset 
of flutter. Aeroelasticity saw one of the earliest uses of model reduction, with 
works appearing in at least as early as the mid-1990s for nonparameterized prob- 
lems [33, 55, 42, 34, 64]. More recently, techniques have been extended to pa- 
rameterized aeroelasticity problems, with the angle of attack and Mach number 
as parameters [47, 46, 4, 2, 5]. We also note that there are nonprojection-based 
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approaches to model reduction, e. g., by the Volterra series; however, given the 
focus of this handbook, we do not cover these works and refer interested readers 
to review papers [26, 49, 25]. 

S4. Model predictive control. Another real-time application of model reduction is the 
control of aerodynamic systems using model predictive control (MPC). Without 
model reduction, MPC is infeasible for large-scale systems, as it requires real-time 
solution of optimization problems. ROMs have been incorporated in MPC to con- 
trol shock location in a supersonic diffuser [36] and to optimize flight path under 
fuel consumption and aeroelastic constraints [3]. 

S5. Uncertainty quantification and state estimation. Model reduction has also been 
used for uncertainty quantification, in which the effect of geometry or flow- 
condition uncertainties are propagated to quantities of interests. ROMs have been 
used for probabilistic analysis of turbine blades, in which simulation is carried 
out for thousands of different configurations [17]. Model reduction has also been 
applied to state estimation, where the aerodynamic flow field is inferred from 
surface pressure tap data [16, 71]. 


6.1.3 Scope and outline 


We make four disclaimers regarding the scope of this chapter. First, we restrict our pre- 
sentation to works on aerodynamics rather than more general fluid mechanics, and 
in particular to works on compressible flow rather than incompressible flow. We re- 
fer to Chapter 9 of this volume for more general coverage of model reduction in CFD. 
Second, given the emphasis of this handbook, we focus on formulation, rather than 
theoretical, aspects of model reduction. We however note that mathematical theories 
have played important roles in the development of model reduction approaches for 
aerodynamics problems; we refer to references provided throughout the chapter for 
further theoretical discussions. Third, the model reduction literature for aerodynam- 
ics problems is vast, with development from both engineering and applied mathemat- 
ics communities; we attempt to cover representative works but admit the coverage is 
not exhaustive and there are inevitable omissions. Fourth, we note that (i) precise re- 
quirements for an ROM depend on the particular engineering scenario and there is no 
universal formulation suitable for all scenarios; (ii) even for a given scenario there are 
many different approaches; and (iii) there are relatively few comparative studies due 
to the recentness of some of the techniques and the shear cost of performing such stud- 
ies for large-scale aerodynamics problems. We hence do not attempt to make definitive 
recommendations and focus on surveying existing approaches, with a hope that the 
chapter will still serve as a guide to construct an ROM that works for the problem of 
interest. 

This chapter is organized as follows. In Section 6.2, we review full-order discretiza- 
tions for aerodynamics problems. In Section 6.3, we review model reduction tech- 
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niques for linearized aerodynamics problems; the linearized problem is relevant for 
small perturbation analysis, which arises in applications including aeroelasticity, flow 
control, and uncertainty quantification. In Section 6.4, we review model reduction 
techniques for nonlinear aerodynamics equations; the full nonlinear analysis is often 
required for aerodynamic shape optimization and flight-parameter sweep. 


6.2 Full-order models 


In this section we review full-order models (FOMs) for aerodynamics problems. We 
consider both the linearized and full nonlinear FOMs; the associated ROMs will be 
constructed in Sections 6.3 and 6.4, respectively. We describe FOMs in abstract forms to 
accommodate various governing equations and discretizations under a unified frame- 
work. 


6.2.1 Conservation laws of aerodynamics 


We introduce the general form of aerodynamics partial differential equations (PDEs) 
considered throughout this chapter. We introduce a P-dimensional parameter domain 
P € R”, ad-dimensional spatial domain Q c RÍ, the associated boundary ðQ, and a 
time interval Z = (0, T] c R. Aerodynamic flow in Q over Z is modeled by a system of 
N, nonlinear conservation laws of the form 


au +V. (u) +f": Vu)) =f™ (u, Vu) inQx T, 


D(u,n-f"(u, Vu) =0 ondQxZ, (6.1) 


Ul;-9 = u? in Q, 


where u is the conservative state, f'"” is the inviscid flux function, f”** is the viscous 
flux function, f*"* is the source function, b is the boundary condition function, and u? 
is the initial state. While the exact forms of the flux, source, and boundary functions 
depend on the specific governing equation — the Euler, Navier-Stokes, or Reynolds- 
averaged Navier-Stokes (RANS) equations - and flow conditions, all conservation 
laws in aerodynamics can be cast in the general form (6.1). We also emphasize that, 
although omitted here for brevity, all functions in general depend on the parameter 
H € P for parameterized problems and the time t € Z for unsteady problems. 

In many aerodynamics problems, our interest is not necessarily in the entire state 
field u but in few quantities of interest (i. e., output). Arguably the most common out- 
put in aerodynamics are lift and drag, which can be expressed as a surface integral of 
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the form 


s= | fo" (u, n- f° (u, Vu); n)ds, 


Thoay 


where Thoday C OQ is the aerodynamic surface of interest, n denotes the unit vector 
normal to Thoqay, and the function f out maps the surface state and viscous flux to aero- 
dynamic forces. 

We make a few remarks about the governing equations in aerodynamics. First, 
inviscid flows are modeled by the Euler equations, which are purely hyperbolic. Sec- 
ond, viscous flows are modeled by the Navier-Stokes equations which, for Reynolds 
number relevant to aerodynamics, are convection-dominated. Third, for turbulent 
flow simulations based on the RANS equations, the Navier-Stokes equations are aug- 
mented with additional empirical PDEs that model the turbulence behavior; most tur- 
bulence models are highly nonlinear, including the one-equation Spalart—Allmaras 
(SA) turbulence model [62] used in most of the works reviewed in this chapter. Fourth, 
nonconservative variables, such as the entropy variables [35], may be used as the 
working state variables; the entropy variables are of particular interest for stability 
analysis of Galerkin methods [12] and in particular ROMs [9, 39]. 


6.2.2 Semi-discrete form 


We now consider a full-order approximation of the conservation law (6.1). While 
there is a number of different discretizations for (6.1), they must provide stability for 
hyperbolic and convection-dominated PDEs. As a result, most works on model re- 
duction for aerodynamics use one the three full-order discretizations: a finite volume 
method [65], a stabilized finite element method [15, 37], or a discontinuous Galerkin 
(DG) method [24, 7]. We refer to the references above for details of the discretizations, 
and here describe FOMs in an abstract form. 

To introduce an FOM, we first introduce a triangulation 7} = {k,,...,Ky,}, where 
TANA is a set of N, nonoverlapping elements such that Q = Uxer, K and k; N Kj = , 
i + j. We next introduce an N;-dimensional approximation space V, c V associated 
with 7p; the associated dual space is denoted by V, with the duality pairing (-,-) : 
V; x V > R. We then introduce an FOM spatial residual operator r} : Vp x P —> V}; the 
particular form of the residual depends on the conservation laws and discretization. 
A semi-discrete form of our FOM problem is as follows: Given u € P, find up(t; y) € Vh, 
t € Z, such that 


OUp(t; p) 


aE Ty(Un(t;)sH) =O in V}, (6.2) 


and u(t = 0; u) = Tpu? (u); here u? (u) e V is the initial condition, and II, : V — Vp is 
a projection operator from V to V,. Throughout this chapter, for any Hilbert space W 
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and the associated dual space W’, the statement “g = 0 in W’” should be interpreted 
as (g,w) = 0 Vw € W. We then introduce an FOM output functional qp : V,xP > R%, 
so that the set of N, outputs is given by 


S(t; H) = 9n(Un(ts WB). (6.3) 


We assume that the solution and output to the FOM exists and is unique. 

We may also consider an “algebraic form” of the problem, i.e., the form of the 
problem described by matrices and vectors, which is convenient for the computational 
implementation of the formulation. To this end, we first introduce a basis ig pi of the 
space V,. We next associate any function v; € V, with a generalized coordinate v, € 
BR: by vp = vg , where A denotes the j-th component of v, and the summation on 
the repeated indices is implied. We then introduce algebraic forms of the FOM residual 
operator ry, : R™ x P > R™, the output functional qh: RN x P > R, and the mass 
matrix M, € IRN» given by 


¥,(W)3 Wi = (r (w g: p), o), i= 1 Np 
Gn (Wis H) = qaw, 9’; y), 
Mri = (PP) bj=b.. Ny 


The algebraic form of the FOM problem (6.2) and (6.3) is as follows: Given y € P, find 
u(t; u) € R™, t € T, such that 
u(t; 4) 


d 
M, Ti r (u,t;p); y) =0 in R™ (6.4) 


for u,(t = 0; 4) = u(y), and then evaluate 


Sp(t; H) = qn (u(t; 4); p). (6.5) 


This algebraic form of the problem is equivalent to the operator form (6.2) and (6.3); 
in particular, up(t; 4) = w (t; Wo . The solution to (6.4) is typically obtained using a 
Newton-like method. 

We make a few remarks. First, for a typical aerodynamics problem, P = O(1 - 10), 
N, = O(10° — 10’), and N, = O(1). Second, for steady problems, the time derivative 
term vanishes and we seek u(y) € V, such that 


r,(u(u);u)=0 in Vý, (6.6) 


or, equivalently, u, (4) € R^» such that r (u, (9); H) = Oin R^». Third, for problems 
with shape deformations, the spatial domain Q depends on the parameter u € D; 
we refer to a review [57] for the treatment of parameter-dependent domains by a 
reference-domain formulation, which provides an equivalent problem in a parameter- 
independent reference domain. Fourth, while finite volume methods are typically not 
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presented as a weak formulation (6.2), the form encompasses (in general high-order) 
finite volume methods, as the methods can be recast as a DG method with an ap- 
propriate state reconstruction function; see, e.g., [13]. Fifth, in any event, all FOM 
discretizations can be expressed in the algebraic form (6.4) and (6.5). Hence, in Sec- 
tions 6.3 and 6.4, we describe all model reduction techniques using this abstract 
framework. 


6.2.3 Full-discrete form 


We now introduce a full-discrete form of the FOM (6.4). We first introduce time in- 
stances 0 = t? < t! < --. < t® = T, and the associated sequence of functions 
ww, = fue, such that u,(t‘;p) = ušu), k = 1,...,K. We then discretize 
the semi-discrete equation (6.4) using a multistep or multistage scheme. For instance, 
if the backward Euler method is used, the full-discrete FOM problem is as follows: 
Given p € P, find tuku) € Ry such that 


rk (UKD: Wh GD) = Mp (uk — uh G) + rauk: H) = 0 

for k = 1,...,K, and uj-°(w) = u(y). Here, tea > R™ x R™ x P > R™ is the full- 
discrete residual operator for the backward Euler method at the time instance k, which 
depends on the state at the previous time step u% (y). More generally, for a multistep 
method, the full-discrete residual operator depends on the states at Kstep previous time 
instances and takes the form ti > RM x RKs x P > R`, We assume that an 
appropriate time marching scheme is chosen such that a sequence of stable solutions 
exists. 

We note that the solution to the steady problem (6.6) is often obtained using a 
pseudo-transient continuation (PTC) method [41], which solves the unsteady prob- 
lem using pseudo-time stepping, to improve the convergence of the nonlinear solver. 
Hence the temporal stability is an important consideration even for steady problems. 
We refer to [41] for a review of PTC methods. 


6.2.4 Linearized equations 


While aerodynamic flow is governed by a system of nonlinear conservation laws, as 
discussed in Section 6.1, time-dependent linearized analysis is also of engineering in- 
terest. The goal of linearized analysis is to propagate small input disturbances to out- 
put perturbations. Here, the input disturbances may result from small changes in the 
geometry (e. g., vibrations), boundary conditions (e. g., gust), or initial conditions; our 
interest is in the associated change in the aerodynamic forces and moments. 

Before we proceed, we make one notational change. In the previous section, we 
introduced the parameter-dependent steady residual operator r} : Vp x P > Vj; in 
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this section, to be consistent with literature on linearized aerodynamics analysis, we 
explicitly separate the parameters subjected to input disturbances from those that are 
not. Specifically, we introduce a Q-dimensional input space Q c R®. We then intro- 
duce the steady residual operator r} : V;, x QxP — Vp, whichis a function of the state, 
input, and parameter. Similarly, we introduce the output operator qp : Vp x Ox P > 
RX, 

In linearized analysis, we decompose the solution up, € V, into a base solution 
up, and perturbation du, so that up = iu, + du,. Similarly, we decompose the input 
v € Qinto a base input v and disturbance ôv so that v = v + dv. The perturbation is 
governed by the following linearized problem: Given u € P and input dv(t) € Q, find 
bu; (t; dv, 4) € Vp, t € T, such that 


oup (t; bv, u) 


7 + J (W 8u (t; dv, u) + B,(W)dv(t) =O in Vj, (6.7) 


and ôu,(t = 0; u) = IIôu? (u) for 6u°(u) € V, the initial perturbation. Here, the Jaco- 
bian J(u) € L(V), Vp) is the Fréchet derivative of r}(, Y; u) : Vp > V; at u,, and the 
operator B,(u) € L(Q, V}) is the Fréchet derivative of r (ŭp =; 4) : Q > V; at v. Given 
the perturbed state ôu,(t; ôv, y) € Vp, we evaluate the associated output perturbation 


ôsp(t; Ov, 4) = Zp(p)ôu,(t; ôv, y), 


where g,(H) € L(V;, R) is the Fréchet derivative of q,(, 0; p) at ii,. The goal of the 
linearized aerodynamics analysis is to map the disturbances in the input v(t) € Q to 
the perturbations in the output 6s),(t; 6v, u) € R™ for any parameter value u € P. In 
aerodynamics, the linearization state u € V, is often the solution to the steady-state 
nonlinear problem (6.6); i. e., r (Ūū; 4) = Oin V}. 

The linearized equations can also be expressed in an algebraic form. To this end, 
we introduce the Jacobian matrix J,(u) € RAM, input matrix B,(M) € RM*2) and 
output gradient vector gą, (4) € R~” such that 


ny = Uns 99’), bj =L. Ny 
B, (Hh); = (B, Üp pe, p`), i = 1, eee Nip j = 1, see Q, 
Sy = (SnUpswee) i= L.. No j= b. Ny 
where æ is the unit vector with the j-th entry equal to 1. The algebraic form of the 


linearized problem is as follows: Given u € P and input dv(t) € Q, find du,(t; dv, u) € 
R~», t € T, such that 


dôu, (t; ôv, 4) 
My, n H 


and evaluate the output 


+ J 9u, (t; ôv, u) + B,(wov(t) =O in R*, (6.8) 


ôs, (t; dv, 4) = Z (p) ôu, (t; ôv, y). 
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We note that, for a fixed parameter u € P, the problem is in the standard linear 
time-invariant form. The application of a time marching scheme yields a full-discrete 
form of the linearized equations whose solution {6u;}{_, satisfies du; = du,(¢*), k = 
1,...,K, analogously to the discussion for the nonlinear FOM in Section 6.2.3. 


6.3 Model reduction for linearized aerodynamics 


In this section we discuss model reduction of linearized aerodynamics problems. As 
discussed in Section 6.2.4, linearized (i. e., small-perturbation) analysis of unsteady 
aerodynamics provides significant insights in many engineering scenarios. More prag- 
matically, model reduction of linearized PDEs requires fewer ingredients than that of 
nonlinear PDEs, and hence we introduce common ingredients in the linearized con- 
text. 


6.3.1 Galerkin method 


We now consider reduced-order approximations of the FOM (6.7) (or equivalently 
(6.8)). To this end, we introduce a sequence of reduced basis spaces Vy- C +- C 
Vy=n,,,,° Cach of which is a subset of V}; for a typical aerodynamics ROM, Nmax = 
O(10-100), which is significantly smaller than N, = O(10°-10’). We then introduce 
the associated hierarchical reduced basis {{" «€ Vi such that Vy = span{¢ NN 
N = 1,...,Nmax- We may also express the reduced basis in an algebraic form {¢” ¢ 
RYH, such that ¢" = ¢ nj g , N = 1,..., Nmax; we introduce the associated reduced 
basis matrix Zy = (¢',...,¢ N) € R™*N, We will discuss various methods to construct 
the reduced basis in Section 6.3.2; for now we assume the basis is given. 

Given a reduced basis space Vy, the semi-discrete form of the Galerkin-ROM prob- 
lem is as follows: Given u € P and dv(t) € Q, find duy(t; ôv, 4) € Vy, t € Z, such that 


oup (t; dv, u) 


a +J (Wun (t; ôv, u) + By(w)dv(t) = 0 in Vy, (6.9) 


and duy(t = 0; u) = Iyu?(u), where Iy : V > Vy is a projection operator from V to 
Vy. Again, for g € V}, the statement g = 0 in Vy should be interpreted as (g,v) = 0 
Vv € Vy. We then evaluate the output perturbation 6s,(t; ôv, y) = Z ()ôuy(t; dv, y). 
The comparison of the FOM problem (6.7) and the Galerkin-ROM problem (6.9) shows 
that the latter results from the restriction of the test and trail spaces to the reduced 
space Vy C Vp. 

The Galerkin-ROM problem (6.9) can also be expressed in an algebraic (or matrix- 
vector) form. To this end, we associate any function vy € Vy with a generalized coordi- 
nate Vy € RY by Vy = vi, ¢ 1; we may also express the full-order generalized coordinate 
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of vy € Vy as V} = vig! = ZyVy € R™, Given the reduced basis, we define the ROM 
operators 


My = ZyM)Zy = (0, Cian at er, 
In) = Zinn = (In, Jian € RYN, (6.10) 
By(H) = ZyIn() = (Bre, G) eta RY 
Sn (H) = $n(W)Zy = CEOE e RYN, 


The algebraic form of the linearized problem is as follows: Given u € P and dv(t; u) € 
Q, find ôuy(t; ôv, p) € RY, t € Z, such that 


My dôuy(t; ôv, u) 


7 + Jy(u)Ouy (ts) + By(w)dv=0 in RY, (6.11) 


and evaluate the output ôsy(t; ôv, u) = gy (Uy (H); ôv, y)ôuy (t; u). Again, the operator 
form (6.9) and the algebraic form (6.11) are equivalent and ôuy(t; y) = dw, (t; we j We 
note that the ROM operators (6.10) are precomputed in the construction stage, so that 
the ROM (6.11) can be solved in O(N") operations for the exponent « between 1 and 3. 
In particular the cost to solve the ROM (6.11) is independent of N,; we recall that N = 
(10-100) and N, = O(10°-10’) for a typical aerodynamics problem. We discuss this 
offline-online computational decomposition in Section 6.3.4. 


6.3.2 Reduced basis for nonparameterized linearized problems 


The efficacy of the Galerkin-ROM (6.9) (or (6.11)) depends on the choice of the reduced 
basis. We now review techniques to identify an effective reduced basis {¢ j A , (or re- 
duced basis matrix Zy € R"), For practical and historical reasons, we first present 
procedures for nonparameterized (or fixed-parameter) problems; the model reduction 
of time-varying but fixed-parameter aerodynamics problems enables fast simulation 
of complex flows, which is essential for, for instance, MPC. As the problems are non- 
parameterized, we suppress the argument ų for all operators throughout this section. 
In addition, as our primary goal is to provide recipes for implementation, rather than 
to discuss theory, we present algorithms in algebraic forms. 


6.3.2.1 Eigenmodes 


A classical approach to identify a reduced basis for linearized aerodynamics problems 
is eigenanalysis. The approach, first introduced by Hall [33], is as follows: 
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1. Solve the generalized eigenproblem: Find the eigenvector ¢ k € R™ and the asso- 
ciated eigenvalue AK € C such that 


J č" =AKM,¢* in R™; 


without loss of generality, sort the eigenpairs such that |A"| > --- > |[A™]. 
2. Construct the reduced basis matrix Zy = (¢',...,¢ N) e REN, 


While historically important, eigenanalysis has a major limitation: The reduced basis 
is based solely on the Jacobian J, and does not account for the system input B, or 
output g,. Hence, the number of eigenmodes N required to achieve a given solution 
or output accuracy is typically greater than empirical approaches based on proper 
orthogonal decomposition (POD). 


6.3.2.2 Time-domain POD 


To address limitations of eigenmodes discussed in Section 6.3.2.1, Romanowski [55] 

proposes a (time-domain) POD approach for linearized Euler equations. We here 

present the method of snapshots [60] to efficiently compute a POD basis for large- 
scale problems in aerodynamics: 

1. Choose L time-dependent training inputs HEr Oher! where l is the training 
input index. 

2. Solve the full-discrete form of the linearized FOM (6.7) for the training inputs 
{6v'}!_, and for K time steps {t*}{_, to construct a snapshot matrix S € RN®*%s, 
whose columns are duk (dv') z ôu, (t; 6v') for k = 1,...,K, 1 = 1,...,L, and 
N, = KL. 

3. Construct the correlation matrix A = $’X,$ in R%:*"s, Here, X, € IRN» such 
that X),; = Cl ; o`) x, is associated with an appropriate inner product; a common 
choice is the L?(Q)-inner product. 

4. Solve the eigenproblem: Find (v*,A*) € R™ x R such that 


Av‘ = Akv in R^; 


without loss of generality, sort the eigenpairs such that |A!| > --- > |A™s]. 
5. Set the reduced basis matrix Zy = t, 325 g^) e RNW where 


GY aA Sv“, k=1,...,N. 
The resulting basis Zy € R™™" is orthogonal with respect to the X, inner product; i. e., 
ZyXnZ y = Iy. In addition, Zy minimizes the X;,-projection error for the snapshots; i. e., 
Zy = arg minw, ern |S - WyWyXSllx,- In this sense, the POD basis is optimal for 
the approximation of the state ôu,(t; ôv) associated with the particular system input 
ôv; however, the system output s, is not accounted for in the POD method. 
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6.3.2.3 Frequency-domain POD 


A variant of the time-domain POD approach above is the frequency-domain POD 
approach proposed by Kim [42] and Hall etal. [34]. As the name suggests, this ap- 
proach takes advantage of the linearity of the problem (6.8) and computes snapshots 
in the frequency domain. Namely, we consider time-harmonic disturbances of the 
form 6v(t) = 6ve of a frequency w € R so that the associated time-harmonic per- 
turbations are of the form ôu, (t; ôv, u) = dt, (S0)e for dt,(50) = GwM), + Ja) Bró, 
where j = V-—1. The frequency-domain POD approach replaces the first two steps of 
the time-domain POD approach in Section 6.3.2.2 with the following: 

T. Choose L training inputs {60! € ROH and K training frequencies {w, € RK. 

2’. Solve the frequency-domain equation 


(jw, My, + J,)5a"(50') = B60! (6.12) 


for {60}; and {wp} to construct a snapshot matrix S € R™™:, whose columns 


are the real and imaginary parts of the frequency-domain perturbation, 
R(6a*(dv')) and 3(6a*(6v!)), for k =1,...,K,1=1,...,L, and N, = 2KL. 


The training input modes and frequencies can be chosen based on known character- 
istics of input disturbances; e. g., for aeroelasticity problems, the modes and frequen- 
cies may be chosen to coincide with the resonance modes of the structure. For lin- 
earized aerodynamics problems, frequency-domain POD is often more efficient than 
time-domain POD and hence is preferred; the approach has been successfully ap- 
plied to the linearized Euler equations in works including [42, 34, 64, 47, 46, 4, 2]. We 
however make two cautionary remarks: First, implementation must support complex 
arithmetic; second, just like time-domain POD, while the POD basis is in some sense 
optimized for the solution field du;,(t; dv) € Vp, it is not specialized for the particular 
system output Ss). 


6.3.2.4 Balanced POD 


The time- and frequency-domain POD approaches construct a reduced space Vy which 
is well suited for the approximation of the entire state ŝu, (t; dv) € Vp; however, in aero- 
dynamics, we are often not interested in the entire state but rather only in few outputs 
(i. e., quantities of interest). In these cases, we can construct a more efficient ROM us- 
ing the balanced POD (BPOD) method proposed by Willcox and Peraire [73], which 
approximates balanced truncation [52] for large-scale problems. The key to BPOD is 
(i) to realize that both the input and output play equally important roles in charac- 
terizing the input-output relationship and (ii) to incorporate the dual problem to ac- 
count for the choice of the output. The dual problem for the linearized aerodynam- 
ics problem (6.8) with a single output (N, = 1) is as follows: Given dv(t) € Q, find 
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Z(t; ôv) € IR%*, t € Z, such that 


= dz, (t; ôv) 
dt 


and then evaluate the output 


+Jjz(t;6v)+g" =0 inR™, (6.13) 


ôs, (t; dv) = 6VBy Zp (ts ôv). 


The associated frequency-domain problem seeks ôŻ(ôv) = (-jwM, HD gi. The BPOD 

procedure based on frequency-domain sampling for N, = 1 is as follows: 

1. Choose L training inputs {60! < RoE and K training frequencies {w, € Ric 

2. Solve the frequency-domain problem (6.12) to collect N, primal snapshots, and 
then obtain the POD mode matrix Z} € RN? and eigenvalue matrix AP e RPP 
for the p > N largest eigenvalues. 

3. Solve the frequency-domain dual problem (6.13) to collect N, adjoint snapshots, 
and then obtain the POD mode matrix Z% € RYP? and eigenvalue matrix AS € 
R”? for the p largest eigenvalues. 

4. Compute the eigenvectors Zy € RN» associated with the N largest eigenvalues of 
the matrix (ZP AȘ Z Dezir AZo" T) using a Krylov subspace method. (Note that 
the matrix is never explicitly formed.) 


The BPOD method produces a reduced basis optimized for the input-output map- 
ping problem and enables goal-oriented reduction of linearized aerodynamics prob- 
lems [73]; depending on the output, BPOD significantly reduces the dimension of the 
reduced space required to achieve a given output tolerance compared to the standard 
POD, as demonstrated for a two-dimensional plunging airfoil [73]. A variant of BPOD 
modified for a problem with a large number of outputs is developed by Rowley in [56]. 


6.3.2.5 Other goal-oriented methods 


We survey a few other goal-oriented methods to generate reduced bases; we again 
restrict ourselves to techniques that have been demonstrated for aerodynamics prob- 
lems. In [74], Willcox etal. propose an Arnoldi-based method, which identifies a re- 
duced basis by matching moments of the FOM input-output transfer function, and 
apply it to aeroelastic analysis of a transonic turbine cascade with unsteady blade mo- 
tions. In [72], Willcox and Megretski propose a method which identifies a reduced basis 
by computing the Fourier expansion of the discrete-frequency transfer function, and 
apply it to analysis of a supersonic diffuser. In [18], Bui-Thanh etal. propose a more 
general approach to goal-oriented model reduction that identifies a reduced basis as a 
solution of a constrained optimization problem and apply it to analysis of a subsonic 
turbine blade. All three methods are goal-oriented in the sense that they consider both 
system inputs and outputs to identify an effective reduced basis. 
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6.3.3 Reduced basis for parameterized linearized problems 


We have so far discussed the construction of reduced bases for nonparameterized 
problems or, equivalently, for one fixed parameter. For parameterized problems, in 
general a reduced basis constructed for one parameter value does not provide a good 
approximation for another parameter value, as the associated dynamics can be very 
different; see, for example, a study for parameterized turbine blades by Epureanu [28]. 
We here discuss a few different strategies to construct reduced bases for parameterized 
problems. 


6.3.3.1 Global POD 


One approach to construct a reduced basis for parameterized problems is to prepare a 

“global” or “composite” POD basis, which has been trained for a range of parameters, 

as proposed for aerodynamics problems by Schmit, Taylor, and Glauser [59, 63]. In this 

approach, we first introduce a training parameter set Ey, = {u™ <a collect the snap- 
shots for all parameter values, and then apply POD to the snapshots. The global POD 
approach for parameterized problem replaces the first two steps of the time-domain 

POD approach in Section 6.3.2.2 with the following: 

T. Choose N, training parameters pry, and L training inputs {óv}. 

2’. Solve the full-discrete form of the linearized FOM (6.7) for the training parameters 
qu” M training inputs {vE and time steps {t* Ka to construct a snapshot 
matrix S € Rs, whose columns are du; (6v'; p”) = ôu, lt“; 6v') fork =1,...,K, 
l=1,...,L,m=1,...,N,, and N, = KLN,. 


The global POD approximation works well for problems with a relatively small pa- 
rameter dimension and extent; however, the method may suffer from two issues if 
the problem exhibits significant parametric variations. First, the FOM may need to 
be solved for a large number of training parameters, which results in a high training 
cost. Second, a large number of POD modes may be required to accurately capture 
the dynamics. (More precisely, if the Kolmogorov N-width of the parametric manifold 
{up (ts ôv, U)}reT,8veQ pep iS large, then a large number of modes is required to achieve 
sufficient accuracy.) 


6.3.3.2 The (weak) greedy algorithm 


To address the potentially high training cost associated with the global POD, the 
(weak) greedy algorithm has been developed [67, 57]. The greedy algorithm suc- 
cessively identifies a reduced basis {¢ Ye based on the behavior of a rapidly com- 
putable error estimate 7, (yu). The algorithm takes as the input the training parameter 
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set &, c D which reasonably covers the domain. Then, in the N-th iteration, given 

Zy_1 € RN“ the algorithm proceeds as follows: 

1. Find the parameter with the largest error estimate: u” = arg MAXycz, Ny-1 (H). 

2. Solve the FOM for u” to obtain u, (uy ye R™, 

3. Augment the reduced basis with the new snapshot: Zy = (Zy Up )); re- 
orthonormalize Zy using Gram-Schmidt. 


The steps are repeated until the user-prescribed error tolerance is met for all y € &,. 
For unsteady problems, Step 3 incorporates an additional reduction technique (e. g., 
POD) to compress the multiple temporal snapshots associated with a single unsteady 
solve; this approach, called POD-greedy algorithm, was proposed and analyzed in [32] 
and its variant is applied to probabilistic analysis of turbine cascades in [17]. 

The weak greedy algorithm has two advantages over POD. First, it requires only 
N FOM solutions compared to N, > N solutions for global POD; hence it reduces the 
training cost, and a larger =, can be used for more exhaustive training. Second, in the 
presence of a goal-oriented error estimate, the ROM trained will meet the error thresh- 
old for the engineering quantities of interest at least for y € =,. However, one major 
limitation of the weak greedy algorithm is that it requires a rapidly computable error 
estimate; due to the difficulty of constructing such an error estimate for hyperbolic 
and convection-dominated problems in aerodynamics, the greedy algorithm has seen 
somewhat limited use in the field. In addition, while the training cost is reduced rela- 
tive to global POD, the resulting ROM may still require a large N if the problem exhibits 
significant parametric variations. We refer to a review paper [57] for more detailed de- 
scription of the weak greedy algorithm. 


6.3.3.3 Parameter-domain decomposition 


One approach to reduce the ROM size for problems that exhibit a large parameter ex- 
tent is to decompose the parameter domain P (or time interval Z) into smaller subdo- 
mains to limit the parameter extent, which in turn controls the reducibility (i.e., the 
Kolmogorov N-width) of the parametric manifold. Namely, we first subdivide P into 
Np subdomains {P"}%” so that UN”, P” = P. We then construct a set of Np reduced 
bases zp for the parametric manifolds { r WDpepr h. To make an ROM predic- 
tion for a given parameter pt € P, we identify the subdomain P” such that u € P” and 
then invoke the ROM. 

One of the earliest applications of the parameter-domain decomposition ap- 
proach in aerodynamics is Annonen etal. [6]; the so-called multi-POD approach 
considers multiple reduced bases associated with different shape deformations. 
Washabaugh etal. [68] also employ the approach for Mach number sweep of a 
full aircraft configuration. Some versions of the reduced space interpolation meth- 
ods [2], which is discussed in Section 6.3.3.4, also incorporates the idea to work with 
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a database of reduced spaces. We also refer to [27] for detailed analyses of parameter- 
domain decomposition approaches. 


6.3.3.4 Reduced-space interpolation based on Grassmann manifold 


Another approach to reduce the ROM size for problems that exhibit a large parameter 
extent is to “interpolate” a set of reduced spaces computed for several parameter val- 
ues to construct a new reduced space for the particular parameter value. One simple 
idea is to interpolate each basis vector ¢ as a function of u € P; however, this ap- 
proach, which works with the vectors and not the space, is shown to work poorly for 
aeroelasticity problems [48]. To address the problem, Lieu et al. [48, 47, 46] propose 
the so-called subspace-angle interpolation method to interpolate any two reduced 
spaces. Subsequently, Amsallem etal. [4, 2] propose a more general approach to in- 
terpolate an arbitrary number of reduced spaces associated with (Zim to construct a 
new reduced basis Zy. The approach builds on the observation that the reduced space 
Vy spanned by a reduced basis Zy is an element of the Grassmann manifold G(N, N,,). 
To interpolate reduced spaces, the approach (i) invokes a logarithmic map to map re- 
duced spaces onto a tangent space, (ii) performs standard interpolation in the tangent 
space, and (iii) invokes an exponential map to map back the logarithmic representa- 
tion of the interpolated basis to identify Zy. Here we outline the algorithm: 

1. Choose parameter values Hin and construct the associated reduced bases 

(zi; i = O is the reference point. 
2. Compute the logarithms {I; € RY? AA given by 


(I= ZyZy")Zy(Zy'Zy) UZV; in Rw, 
T; = Utan (£V? in RN, 


where the right-hand side of the first stepis the thin singular value decomposition 
of the matrix in the left-hand side. 

3. Given yu € P, interpolate each entry of the parameter-logarithm-matrix pairs 
qi, rye using a multivariate interpolation scheme for R? to find T € RN 
associated with ų € P. 

4. Compute the exponential map of the logarithm T € R" given by 


r = Uxv’, 
Zy = ZyV cos(Z) + Usin(2). 


This interpolation method on the Grassmann manifold can be thought of as a general- 
ization of the subspace-angle interpolation method [48, 47, 46]; the two methods are 


6 Model reduction in computational aerodynamics —— 217 


equivalent when Nz = 2 reduced bases are used for interpolation, but the former gen- 
eralizes to an arbitrary number of reduced bases [4]. For problems with a large param- 
eter extent, the reduced space interpolation methods can also be combined with the 
parameter-domain decomposition method discussed in Section 6.3.3.3; in this case, 
the interpolation is performed on a subset of all available reduced bases [2]. The re- 
duced basis interpolation methods have been demonstrated for parameterized aeroe- 
lastic analysis of full aircraft configurations [4, 2] as discussed further in Section 6.3.6. 


6.3.4 Offline-online computational decomposition 


As briefly discussed in Section 6.3.1, model reduction achieves computational speedup 

through offline-online computational decomposition. The offline stage is expensive 

but is performed only once. The online stage is cheap, and it is invoked in real-time 

for many different inputs and/or parameters. To describe offline-online computational 

decomposition for linearized aerodynamics problems, we break down the model re- 

duction procedure into three steps: 

1. Collect the FOM snapshots and construct a reduced basis Zy (or reduced bases 
{Zġ) using a method described in Section 6.3.2 or 6.3.3. 

2. Construct the ROM operators by projecting the FOM operators onto the reduced 
basis Zy according to (6.10). 

3. Given the input v(t) € Q, t € T, solve the ROM problem (6.11). 


In general, Step 1 is the most expensive stage, as it requires time- or frequency-domain 
solutions of the FOM for a number of different control inputs and/or parameters. Step 2 
also requires access to the FOM, and hence does require O(N,,) operations; however, 
this step is much cheaper than Step 1, as performing the projection (6.10) is much 
cheaper than solving the FOM (6.11). Step 3, which works exclusively with the ROM, 
requires O(N") operations; since N} = O(10° - 10’) and N = O(10 - 100) for a typical 
aerodynamics problem, the ROM achieves significant computational reduction rela- 
tive to the FOM. 

The offline-online computational decomposition takes on different forms depend- 
ing on whether the problem is parameterized. For nonparameterized problems, the of- 
fline stage comprises Steps 1 and 2; first a reduced basis is identified using a method in 
Section 6.3.2, and then the ROM is constructed in terms of the reduced operators (6.10). 
In the online stage, given an input dv(t) € Q, t € Z, we invoke the ROM (6.11); note that 
the online stage requires only O(N") operations. 

For parameterized problems, the offline stage comprises only Step 1; either a 
global reduced basis or a set of reduced bases is constructed using a method in Sec- 
tion 6.3.3. In the online stage, given u € P, we first identify an appropriate reduced 
basis: For the parameter-domain decomposition method discussed in Section 6.3.3.3, 
this step requires the identification of the subdomain P” to which p belongs; for the 
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reduced space interpolation method discussed in Section 6.3.3.4, this step involves 
the interpolation of the reduced bases. We then perform Step 2; project the FOM op- 
erators onto Zy to identify the ROM operators (6.10). We finally invoke the ROM to 
approximate the linearized aerodynamics problem for the given u € P and dv(t) € Q, 
t € T. Unlike the online stage for nonparameterized problems, the online stage for 
parameterized problems requires the access to the FOM in Step 2 and hence requires 
O(N,) operations. Nevertheless, significant speedup can be achieved relative to the 
FOM as Step 2 is still much cheaper than the unsteady solution of the FOM. 

We note that if the parameterized FOM operators admit a decomposition that is 
affine in functions of parameters, then the associated reduced operators can be pre- 
computed in the offline stage and hence the online cost would be O(N"); however, 
most of the relevant problems in aerodynamics do not admit this so-called affine pa- 
rameter decomposition. We refer to a review paper [57] for offline-online computa- 
tional decomposition in the presence of affine parameter decomposition. We also note 
that it may be appropriate to invoke the empirical interpolation method [10, 31] or its 
variant to identify an approximate affine decomposition; see Chapter 5 of Volume 2. 


6.3.5 Stability of the Galerkin-ROM 


As discussed in Section 6.1, the focus of this handbook is on formulation and not the- 
ory. However, as time stability of ROMs (6.7) is one of the key issues in model reduc- 
tion of linearized aerodynamics problems, we briefly mention relevant literature; here, 
time stability refers to the ability to bound some norm of the solution ||u(t)||, by the 
initial state and boundary conditions. 

Barone etal. [9] and Kalashnikova etal. [39] analyze the time stability of the 
Galerkin-ROM (6.9). The works show that the ROM is stable if the symmetrized form 
of the hyperbolic system is used with appropriate boundary conditions. We note that 
for compressible Euler and Navier-Stokes equations (i) the symmetrized system is 
described in the entropy variables [35, 11]; (ii) the associated energy norm is given by 
(w,v)a = J 5 v’ Ajwdx, where A, is the Jacobian of the conservative variables with 
respect to the entropy variables; and (iii) the mass matrix in (6.7) is also modified 
accordingly. Kalashnikova et al. [40] further extend the stability analysis to aeroe- 
lasticity problems where the structure is modeled by a linearized von Karman plate 
equation. 

In addition to analysis, we note there are ROM formulations that are designed 
to achieve guaranteed stability; we again restrict ourselves to works that have been 
demonstrated for aerodynamics problems. The Fourier-based formulation of Willcox 
and Megretski [72] discussed in Section 6.3.2.5, for instance, is guaranteed to preserve 
stability of the underlying FOM; the method has been applied to model reduction [72] 
and MPC [36] for which POD yields unstable ROMs. Amsallem and Farhat [5] also pro- 
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pose an online-efficient stabilization based on Petrov—Galerkin projection and apply 
it to aeroelastic analysis of a wing-store configuration. 


6.3.6 Large-scale applications 


We conclude this section on model reduction for linearized aerodynamics problems 
with a few applications to large-, industry-scale problems. 


Aeroelastic analysis of the AGARD model 445.6 wing [64]. In this work Thomas et al. 
consider flutter prediction of a weakened AGARD model 445.6 wing. The FOMs 
consist of N, = 2.6 x 10° to 7.8 x 10° aerodynamic degrees of freedom. The flutter 
boundaries for six different values of the base flow Mach number are analyzed. For 
each flight Mach number, the snapshots are computed for the first five structural 
resonance modes ({6¢'}) and six frequencies ({w“}); POD is applied to identify a 
ROM with N = 55 modes. The ROM is then used to construct the root loci with 
respect to the reduced velocity and to provide accurate predictions of flutter ve- 
locities. 

Aeroelastic analysis of a full F-16 aeroelastic configuration [2]. In this work Am- 
sallem et al. consider model reduction of a full aeroelastic F-16 configuration. The 
FOM consists of N, = 2 x 10° + 1.7 x 10° aerodynamic and structural degrees of 
freedom, respectively. The parameters are the base flow Mach number and angle 
of attack. In the offline stage, a set of reduced bases for 83 different flight con- 
figurations are prepared using the frequency-domain POD approach; each basis 
comprises N = 90 modes. In the online stage, the reduced bases are interpolated 
on a manifold as discussed in Section 6.3.3.4. For the five predictive test config- 
urations considered, the error in the L*(Z)-norm of the unsteady lift varies from 
0.4% to 7 %. The time to solve the linearized system is reduced by a factor of 90 in 
the online stage. (However, the online stage also requires the computational of the 
steady-state equilibrium solution; when this step is taken into account, the overall 
speedup factor is approximately 7.) The aeroelasticity problem is also considered 
in [47, 46, 4]. 

Probabilistic analysis of unsteady turbine blades [17]. In this work Bui-Thanh et al. 
consider model reduction of a two-blade turbine system to analyze the effect of 
geometric uncertainties on unsteady lift forces. The FOM consists N} = 1 x 10° 
degrees of freedom. The geometric modes are identified using principal compo- 
nent analysis on data from 145 real blades; geometric perturbations are parame- 
terized using P = 10 parameters. The reduced basis are identified using a greedy 
algorithm modified for the high-dimensional parameter space; the resulting ROM 
consists of N = 290 modes. The reduced model is then invoked for 10,000 differ- 
ent geometries to estimate the distribution of the work per cycle (WPC). Relative 
to the FOM, the ROM achieves less than 0.5 % error in the mean and 2% error in 
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the variance. The time to complete the 10,000 analyses is reduced from 516 hours 
for the FOM to 1.1 hours for the ROM, a computational reduction by a factor of 468. 


6.4 Model reduction for nonlinear aerodynamics 


In this section we discuss model reduction of nonlinear aerodynamics problems. 
While linearized analysis suffices for some aerodynamics scenarios, applications 
such as shape optimization and flight-parameter sweep require full nonlinear analy- 
sis. As some of the model reduction ingredients are the same as those discussed for 
linearized problems in Section 6.3, we focus on techniques and challenges that are 
unique to full nonlinear analysis. 


6.4.1 Projection methods 


While the Galerkin method is by far the most common approach for model reduction of 
linearized aerodynamics problems, there are a few different projection methods that 
are commonly used for nonlinear aerodynamics problems. We here review the two 
most popular methods, the Galerkin and minimum-residual methods, and provide a 
short discussion of other approaches. 


6.4.1.1 Galerkin method 


We first introduce the Galerkin approximation of the nonlinear aerodynamics prob- 
lem (6.2). As in Section 6.3.1, we assume that a sequence of reduced basis spaces Vy_, C 
‘++ C Vy-y,,,, and the associated hierarchical reduced basis {¢ j pa is given; we discuss 
the procedures to generate the reduced basis in Section 6.4.3. The semi-discrete form 
of the Galerkin-ROM problem is as follows: Given py € P, find uy(t; 4) € Vy, t € Z, such 
that 


OU y (t; u) 


a” Tr(Un(ts);H@) =O in Vy, (6.14) 


and uy(t = 0; u) = ITpu?(p). Again, for g € V}, the statement g = 0 in Vy should be 
interpreted as (g, v) = O Vv € Vy. We then evaluate the output sy(t; 4) = qa (uyn (t; u); 4). 

We may also consider the algebraic form of the problem. We recall from Sec- 
tion 6.3.1 that we associate any function vy € Vy with a generalized coordinate 
Vy € RY by Vy = vi) ; we may also express the FOM generalized coordinate of 
Vy € Vy aS V} = vig) = ZyVy € R™, Given the basis, we define the ROM residual 
ry : RY x P > RY, output functional qy : R x P > R, and mass matrix My € RY” 
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such that 


ry (WysH) = Zit, (Zywys W) = (rn (whys u), Cio 


Gy (Wy5H) = An(Zy Ws) = qa(W S's Ht), 
ERN N 
My = ZyM),Zy = (7, S)pe@ijr 


The algebraic form of the Galerkin-ROM problem is as follows: Given u € P, find 
uy(t;u) € RY, t € Z, such that 


dux(t; p) 
dt 


+ry(uy(t;iu);p)=0 in RY, (6.15) 


My 

and uy(t = 0;p) = u? (p), where u? (u) € R” is the generalized coordinate for Tyu? (u). 

We then evaluate the output sy (t; 4) = y(Uy(t; y); 4). The operator form (6.14) and the 
algebraic form (6.15) are equivalent in the sense that uy(t; 4) = oe w (t; we. 

Most aerodynamics shape optimization and flight-parameter sweep scenarios 

consider steady-state solutions. The steady-state problem seeks uy (4) € Vy such that 


ry(uyQo;w)=0 inR, (6.16) 


and then evaluates sy(H) = qy(Uy (H8); H). 

We make a few observations. First, the reduced-order Galerkin problem (6.14) 
(or (6.15)) is in semi-discrete form; as described for FOMs in Section 6.2.3, we ap- 
ply a suitable time marching scheme to obtain a full-discrete form of the Galerkin- 
ROM problem. Second, the steady-state problem (6.16) is solved using a pseudo- 
time continuation method as discussed for FOMs in Section 6.2.3, and hence the 
unsteady equations are relevant also for steady-state problems. Third, although the 
approximation space Vy is of dimension N, the computation of the reduced residual 
ry(Wy; H) = Z’ r (ZyWy; H) requires O(N,) > O(N) operations, because the FOM 
residual r,(ZyWy;Ņ) € R™ must be projected onto the reduced basis Zy € R®®Y, 
Hyperreduction, which enables O(N) evaluation of the residual, is discussed in Sec- 
tion 6.4.2. 


6.4.1.2 Minimum-residual method 


We now discuss an alternative projection method: the minimum-residual method. As 
the name suggests, we choose the element of Vy that minimizes the (dual) norm of the 
residual as our ROM solution. For steady problems, the minimum-residual problem is 
as follows: Given p € P, find uy (H) € Vy such that 


3 p ru 3H), Vv 
Uy(H) = arg infr (uv (W: p) lv; = arginf sup Talun W: H) Vn) (6.17) 
wy €Vy 


wWyeVy VhEVn IIVally, 
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An algebraic form of the problem is as follows: Given p € P, find uy(y) € R” such that 


uy (HW) = arg inf|r,(ZywysW|\w, (6.18) 


WyeR 


= arg inf ty(Zy Wy; U)” Wpr,(ZyWy; H), 
wyeR” 


where W, € IR“ is the inner product matrix; the choice W, = v; for Vai = 
(g’, Pv, i,j = 1,..., Ny, results in (6.18) to be equivalent to (6.17). 

The minimum-residual formulation can also be extended to unsteady problems 
as follows: Given p € P, find uy(t;p) € RY, t € I, such that 


uš (u) = arg infr a(ZyWy; VARTI) lwp K=1...K. 
wy eR 

The formulation minimizes the residual associated with each time step. 

We make a few observations. First, the minimum-residual method can be cast as 
a Petrov—Galerkin method [50]; as a result, the method is also referred to as a least- 
squares Petrov—Galerkin method [21]. Second, the minimum-residual method is a very 
common approach for model reduction of steady nonlinear aerodynamics problems 
and has been used in works including [43, 44, 45, 69, 80, 79]. Third, similarly to the 
Galerkin method, the evaluation of the FOM residual in (6.18) requires O(N hj > O(N) 
operations. Hyperreduction, which enables O(N) evaluation of the residual, is dis- 
cussed in Section 6.4.2. 


6.4.1.3 Other approaches: interpolation- and L'-based ROMs 


While the Galerkin and minimum-residual methods are most commonly used meth- 
ods for model reduction of nonlinear aerodynamics problems, some works have used 
interpolation-based ROMs, which deduce the reduced basis coefficients uy € RY 
through interpolation. In the context of aerodynamics, the approach has been ap- 
plied to flight-parameter sweep scenarios: Bui-Thanh etal. [16] deduce the reduced 
basis coefficients using cubic splines for two-dimensional Euler flow over an air- 
foil; Franz et al. [30] deduce the reduced basis coefficients using a manifold learning 
technique for three-dimensional Euler flow over a wing. 

We can also consider minimization of different norms of the residual to deduce 
Uy € R”. Of particular interest is the L!-norm, which is a more natural norm for hyper- 
bolic equations. Based on this observation, Abgrall and Crisovan [1] propose an ROM 
which identifies the solution through L'-minimization and apply it to parameterized 
transonic Euler flow over an airfoil. 
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6.4.2 Hyperreduction 


As discussed in Section 6.4.1, seeking the solution in a reduced space Vy c V} is in- 
sufficient to achieve O(N’) online cost for nonlinear problems. We need a means to 
approximate the projection of the FOM residual r,(wy; 4) € R™ onto the reduced ba- 
sis Zy € IRN" in O(N) operations for the Galerkin method, and there is an analogous 
requirement for the minimum-residual method. This is the goal of hyperreduction, a 
term coined by Ryckelynck [58]. We here present hyperreduction approaches that have 
been used for aerodynamics problems; we refer to Chapter 5 of Volume 2 for a more 
general coverage. We follow the convention used in much of the hyperreduction liter- 
ature and present formulations in algebraic form. 


6.4.2.1 Minimum-residual collocation methods 


We first consider arguably the simplest hyperreduction method: the minimum-residual 
method with a collocation-based approximation of the residual norm. To begin, we 
assume that the FOM residual can be decomposed into elemental contributions; 
the assumption holds for finite volume and finite element methods — the two most 
commonly used discretizations in aerodynamics — as the FOM residual is assembled 
element by element. We express this elemental decomposition of the residual as 


Ne 
Yr (W; H) = $ Yhx(Wp H) in R”, 


K=1 


where N, = |7;| is the number of elements and r} x : R\: xP — R™ is the FOM residual 
operator for the x-th element. Note that r} x(W; H) € R™ is mostly sparse, because a 
given element contributes to a small number of residual degrees of freedom. 

We now proceed with hyperreduction. We first choose a small subset of Ñ, sam- 
ple elements f, c Tp so that N < Ñ, « N,; we denote the associated sample element 
indices by T. (Quantities associated with hyperreduction bear 7 throughout this sec- 
tion.) We then consider the following hyperreduced approximation of the minimum- 
residual problem (6.18): Given u € P, find ñŭy(H) € RW such that 


ŭy (H4) = arg min 
wy eR” 


È hel Wai w| : (6.19) 
KET, 2 


We observe that if N, = O(N) « Nj, then we can solve this hyperreduced minimum- 
residual problem in O(N") operations. 

We can also describe the hyperreduction procedure algebraically. To this end, 
we first identify the set of Ñ residual sample indices Z = fbig) associated 
with the sample elements 7;,. For finite volume methods, Ñ+ = N.N, as the number 
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of residual degrees of freedom associated with each element is equal to the num- 
ber of components N, in the PDE. We then introduce the associated sample matrix 
P= (et, ...,e"2) e RM*NZ whose j-th column is the canonical unit vector eù e RM, 
The minimum -residual collocation problem (6.19) is equivalent to 
üy (p) = arg min||P r, (wy; Wl- 
wyERN 

Here, to achieve hyperreduction, we evaluate the operator (P’r,) : R xP > RNz by 
first checking which indices are requested by P and then computing the residual for 
only those indices. 

The key to a successful hyperreduction by the minimum-residual collocation for- 
mulation lies in the selection of the sample elements 7;,, which is performed in the 
offline stage. We here review a few approaches that have been applied to aerodynam- 
ics problems. 

Physics-informed selection. To our knowledge, LeGresley and Alonso [44] were the 
first to consider hyperreduction for aerodynamics problems. In the work, hyperreduc- 
tion is achieved by including only 20 %-30 % of the elements near the airfoil in fp. 
This strategy was specialized for aerodynamic shape optimization, in which most of 
the solution variations are in the vicinity of the airfoil. Vendl et al. [66] also consider a 
physics-informed hyperreduction in the context of flight-parameter sweep; however, 
as the parameter affects the boundary conditions, they also included elements on the 
far-field boundary in 7;,. 

Gappy POD on the state snapshots. To devise a more systematic approach to iden- 
tify sample elements, Washabaugh etal. [69, 70] invoke gappy POD [29] on the solu- 
tion snapshots S = (u,(u’), a „u (ws) e RNs and set the sample indices Z for 
the minimum-residual collocation method equal to the gappy POD sample indices. 
Specifically, the method successively processes sets of snapshots S € IRN**s in smaller 
batches S% = (u),(u’), ... upu“) € RN k = 1,..., N; assuming the sample indices 
T have been constructed for $;,_,, the sample indices are updated for the batch $% as 
follows: 

1. Compute the gappy POD reconstruction of the snapshots: 
Š; = Zy(P’Zy)'P'S € RNs, Here, (-)* denotes the pseudo-inverse. 
2. Set i* = arg maxicty,) MaXjej,N,] ISK - Sxlij- 
3. Add the sample index: Í = Z Ui*; update the sample matrix P accordingly. 


This approach assumes that the sample indices with which the state can be approxi- 
mated work well also for the residual; this assumption allows the method to work with 
the state and not the residual, which significantly reduces the offline cost relative to 
Gauss— Newton approximate tensor (GNAT) and empirical quadrature procedure (EQP) 
methods discussed in Sections 6.4.2.2 and 6.4.2.3, respectively. The method has been 
applied to full aircraft configuration under shape deformations [69] as discussed fur- 
ther in Section 6.4.5. 
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6.4.2.2 Gauss—Newton approximate tensor method 


The GNAT method [21, 22] approximates the minimum-residual problem (6.18) for 
W, = I, uy(}) = arg infy, er" [lt_(ZyWy; M)llz, using a gappy POD approximation [29] 
of the residual and Jacobian and then solves the problem using the Gauss—Newton 
method. (Although the original work [21, 22] considers unsteady problems, for no- 
tational simplicity we here consider a steady problem.) The solution of (6.18) by the 
Gauss—Newton method requires successive solution of the linear least-squares prob- 
lem: Find the update wy € RY such that 


OWy = arg min|Ja(ZyVy)ZnVy + ¥),(ZyWy)|[- (6.20) 
VER’ 


The solution is then updated according to wy — Wy + adwy, where the step length 
a € (0,1] is deduced by line search. The cost to solve this least-squares problem is 
O(N,,) as it requires the FOM residual and Jacobian. 

To approximately solve (6.20) in O(N) operations, the GNAT method prepares 
three ingredients for a gappy POD approximation of the residual r, : R` xP > R™: (i) 
a reduced basis for the residual Z” € R™*™ , (ii) a set of sample indices Z = { ip... NA 


for Nz > N,, and (iii) the associated sample matrix P = (e, i3 er) e Reiz whose 

j-column is the canonical unit vector et € R». The residual is then approximated by 

regression: f}(ZyWy) = arg Minyev, IP" (r,(Zywy) - Z’v)||>. The Jacobian is similarly 
approximated using a reduced basis for the Jacobian Z’ € R™™™ and the same sam- 
ple matrix P" by regression: Jn(Zywy)Zy;j = arg MiNyey, |P7 n(ZyWy)Zyj - Zv)\lo, 

j = 1,...,N. The GNAT method solves this gappy POD-approximated minimum- 

residual problem using a gappy POD-approximated Gauss—Newton method. 

Carlberg et al. [21, 22] introduce four variants of the GNAT method, named proce- 
dure 0-3. We here consider only procedure 1, which has been shown to exhibit good 
accuracy and robustness for unsteady aerodynamics problems. We outline the offline 
and online stages of the GNAT method. 

Offline stage. In the offline stage, we construct all ingredients of GNAT: Zy € 
RYN 2? =Z e RN and P e RZ, 

1. Choose a snapshot parameter set £, = eye, cP. 

2. Solve the FOM (6.4) for each y € E, to obtain {u,(#)},,<z,. Apply POD to the snap- 
shots to obtain a state reduced basis Zy € IRN, 

3. Solve the nonhyperreduced ROM (6.17) for each u € =. Collect residual snapshots 
{t,(ZyUy(#); H)}uez,: Apply POD to the set to obtain a residual reduced basis Z” € 
RYN for N, > N. Set Z’ = Z’. 

4. Apply the gappy POD procedure described in Section 6.4.2.1 (for the state snap- 
shots) to the residual snapshots to determine the sample index Z with Nz > N, 
and the associated sample matrix P € RYN, 

5. Precompute A = (PTZ)! € RN*%z and B = (ZIZ (PTZ)! € RVM, 
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Online stage. In the online stage, given u € P, we seek uy (H) such that 


Uy(#) = arg min|Z’ (P7Z")' Pr, (Zywyy; W| 


wyeR” 


This problem is solved using the Gauss—Newton method as follows: 
1. Form C(wy) = P’J,(ZyWy)Zy and D(wy) = P’r,(Zyw). 
2. Solve the linear least-squares problem: Find 6wy € R” such that 


ôwy = arg min|AC(wy)v + BD(wy)||,. 
veRY 


3. Update wy — wy + adwy, where a is determined from line search. 
4. Ifconverged, terminate; otherwise return to 1. 


The online computational cost is O(N’) and is independent of the FOM. To evaluate 
the output, the GNAT method does not explicitly hyperreduce the output functional 
qh: RX x P > R, but simply leverages the fact that output functionals for most aero- 
dynamics problems require evaluation on a small subset of elements, e. g., elements 
on aerodynamics surfaces. Hence, output evaluation constitutes a small fraction of 
the overall cost. 

The GNAT method has been applied to large-scale simulation of (nonparameter- 
ized) unsteady turbulent flow over the Amhed body [21, 22] as discussed further in 
Section 6.4.5. We also refer to [20] for a detailed analysis of the method. 


6.4.2.3 Galerkin method with empirical quadrature procedure 


One of the limitations of the hyperreduction methods discussed in the previous two 
sections is that they do not provide a quantitative control of the solution and/or out- 
put error due to hyperreduction. One approach which provides such quantitative error 
control is the EQP [77, 75, 76]. To describe the method, we first introduce the hyperre- 
duced residual ty : RY x P > R and output functional qy : RY x P > R of the 
form 


Ne Ne 

ty (Wy) = È prfnx(Wy; H) = YZ nth «(Zn Wy3 H), (6.21) 
K=1 K=1 
Ne Ne 

ån(Wr: H) = È plan (ws W = È play y(Zy Wyss (6.22) 
K=1 K=1 


here p” € R? and p1 € IR“ are the EQP weights that are sparse (i. e., most entries are 
zero) so that the summands need to be evaluated for a small subset of elements. The 
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associated hyperreduced problem is as follows: Given py € P, find y(t; 4) € RY,t eZ, 
such that 


ani) +Ëy(ŭy m(t W:u)=0 in R", 


My 
for Uy(t = 0; y) = u? (p), and evaluate the output šņy(t;Ņ4) = y (ŭy(t; p); 4). We wish 
to find EQP weights p” € IR“ and p? € R™ so that (i) Isy(t; u) - ŝy(t;p)| < 6 for a 
user-prescribed tolerance 6 € R,, and (ii) nnz(p”) = O(N) and nnz(p%) = O(N). The 
two conditions ensure the accuracy and online efficiency, respectively, of the hyperre- 
duced ROM. 

The EQP weights are computed in the offline stage by solving linear programs 
(LPs). We first introduce a parameter training set ©, = {pe AG and the associated train- 
ing states U, = {Uy(H)}yex,. The training states can be the nonhyperreduced ROM solu- 
tion as it is done for GNAT; however, when used in conjunction with the greedy algo- 
rithm, U, can be the hyperreduced ROM solution in a given iteration [75]. The general 
form of the linear program, denoted LP’, where * is the placeholder for the residual “r” 
or output function “q,” is as follows: Find the basic feasible solution p** € R? such 
that 


Ne 
p”* = argmin Ý pi, 


preIRNe K1 


subject to nonnegativity constraints 


and manifold-accuracy and constant-integration constraints 


aj(u') Ne ap, Q’) ; b` (ut) 5 
: , : Pı : : 
E EE ; Ss Mi + "|. (6.23) 
aus) ay (uN) J? pg’) 5 
PN, — 
xl gis ky, | IQ] ôo 


where a,(H) € RN, x =1,... , No, is a set of vectors that depends on the specific man- 
ifold accuracy constraint to be described shortly, N} is the number of constraints per 
training parameter, b'(u) = ys a,(H) € BN, 6° € RY is the manifold-accuracy tol- 
erance, |x| = f dx, and |Q| = fo dx. The LP can be solved using a simplex method. 
We now introduce specific manifold accuracy constraints for the residual (6.21) and 
output functional (6.22). 

Residual EQP. The residual EQP weights p” <€ R”: are found by solving LP”(E;, 
U,, 6"). As our goal is to control the output error, we introduce a reduced basis approx- 
imation of the dual problem: Given u € P and the linearization state uy(4) € RY, find 
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the dual solution zy(u) € R" such that 


Jy (Gy (H); H) ZW) = Sv Qos) in R”. 


As discussed in the context of balanced POD in Section 6.3.2.4, the dual solution re- 
lates the residual to the output error. The manifold-accuracy constraint (6.23) for the 
residual imposes N% = N constraints per training parameter given by 


ap) = |Z) ° [ivp] in RY, 


and 8’ = iy, where 1y € RY is the vector of all ones and © is the Hadamard (i.e., 
entrywise) product. Overall, LP” has N, unknowns, N, nonnegativity constraints, and 
2(N,N + 1) inequality constraints (where the leading factor of two accounts for the 
upper and lower bounds in (6.23)). 

Output functional EQP. The output EQP weights p° € IR are similarly found by 
solving LP (E; U;, 6%). The manifold-accuracy constraint (6.23) for the output func- 
tional imposes N4 = 1 constraint per training parameter given by 


af (u) = qx(ŭy(p); 4) inR. 


Overall, LP? has N, unknowns, N, nonnegativity constraints, and 2(N; + 1) inequality 
constraints; the LP for the output functional is much smaller than that for the residual. 

Output a posteriori error estimate. The EQP method also provides an a posteriori 
error estimate for the output error. The error estimate is based on the dual-weighted 
residual method [14]. To this end, we first introduce a separate reduced basis for the 
dual problem zo € RMN, which is different from the primal reduced basis Zy € 
R™:*N We then introduce an EQP approximation of the residual, Jacobian, and output 
gradient evaluated with respect to the dual reduced basis zau, i : R xP RY, 
jee: RY x D > RYN and g% : RY x D — RN such that 


Ne Ne 
iy (W; 4) = D Patni (Zy Ws M) = D PRE fhx(ZyWy; H), 
K=1 K=1 


Ne N: 
IN (wp) = Y py (Zywys) = > oZ Jnx(ZNWy:; ZN 
K=1 


K=1 


Ne Ne 
EN (ws) = > pig y (Zywys) = > pray Sax (ZW Ms 


K=1 K=1 


for some EQP weights p” € R^: computed in the offline stage. The EQP dual problem 
is as follows: Given u € D and ay(p) € RY, find zdu (p) € R” such that 


jag: p) = 8" üy) in R”. 
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The output error estimate is given by 


fin W = ži (wt (ay GO); p)|. 


Assuming nnz(p") = O(N), this error estimate is computable in O(N) operations. 

The output error estimate EQP weights p” € R^: is given by a linear program 
LP"(&,, U, 67). The manifold-accuracy constraint (6.23) for the output error estimate 
imposes N? = 3N constraints per training parameter given by 


max{|zy hoe aia o [tN xe P| 
al (uw) =| max{iri han aa OEC in R™; 


here zł% = (v8"/N)"?/2 and Yimin = (8"/(UN))"?/4 for v = [z0r], the 
maximum operator is taken entrywise, and all entities with the argument p are eval- 
uated about the state u(j) and the parameter p; e. g., ry (HM) = ry (Hy (H); u). Overall, 
LP” has N, unknowns, N, nonnegativity constraints, and 2(3N,N + 1) inequality con- 
straints. 

The EQP method has been applied to two- and three-dimensional turbulent aero- 
dynamic flows in the context of flight-parameter sweep [75, 76]. The rapidly com- 
putable output error estimate enables the construction of a reduced model that meets 
the user-prescribed error tolerance in an automated manner in the offline stage and 
provides reliable predictions in the online stage. 


6.4.2.4 Choice of a hyperreduction procedure 


We make a few remarks about the choice ofa hyperreduction method for aerodynamics 
problems. One of the challenges in hyperreduction for aerodynamics is that the FOM is 
typically very large, with millions of degrees of freedom, and hence the offline training 
cost cannot be neglected in a practical engineering setting. This is unlike some clas- 
sical model reduction scenarios, where the offline cost is often neglected. The other 
challenge is the stability; the hyperreduced system must provide time stability for un- 
steady simulations to produce meaningful results and for steady simulations to find 
solutions using the PTC procedure. There exist many examples in the literature where 
a hyperreduction method that works well for other nonlinear problems has been found 
to be insufficient for aerodynamics problems. 

For instance, the missing point estimate [8] chooses the sample indices Z such 
that the associated sample matrix P minimizes the condition number of ZT PP"Z N3 
however, the method was deemed too expensive for steady aerodynamics problems 
in [66]. The empirical interpolation method [10, 31] and its discrete counterpart [23], 
which are arguably the most common hyperreduction methods, to our knowledge have 
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seen limited use in aerodynamics; in fact, Carlberg et al. [22, 20] report temporal insta- 
bility for turbulent unsteady flows. Similarly, the GNAT method, which has been used 
successfully for nonparameterized unsteady problems, was deemed too expensive for 
parameterized steady aerodynamics problems in Washabaugh [70]; we also refer to 
the work for detailed discussion of the choice of a hyperreduction method. 


6.4.3 Construction of reduced basis 


Techniques to find an appropriate reduced basis for nonlinear aerodynamics prob- 
lems are largely the same as those for linearized aerodynamics problems discussed 
in Sections 6.3.2 and 6.3.3. By far the most popular method to generate reduced bases 
for nonlinear aerodynamics problems is POD [43, 44, 45, 69, 80, 79, 66, 21, 22]. For 
unsteady problems, the snapshots are collected for K time steps to yield S = fuk = 
w(t); for parameterized problems, the snapshots are collected for N, training pa- 
rameters £, = ONA to yield S = {u (H)}„ez, Given the snapshot matrix S, the POD 
procedure to identify Zy € IRN»* is described in the context of linearized problems 
in Section 6.3.2.2. For the EQP method which provides an online efficient a posteriori 
error estimate, it is also possible to identify the reduced basis using the weak greedy 
algorithm discussed in Section 6.3.3.2 [75, 76]. We note that while the “standard” POD 
readily extends to nonlinear problems, some of its variants which rely on the linearity 
of the PDE, such as frequency-domain POD or balanced POD, do not. 


6.4.4 Treatment of moving discontinuities 


One of the challenges in model reduction of transonic aerodynamics problems is 
the treatment of shocks. The fundamental challenge is that if u,(t; u) contains a 
discontinuity whose location depends on t € Z orp e€ P, then the Kolmogorov 
N-width of {uy,(t;#)}:ezep is large and the solution manifold is not amenable to 
a low-dimensional approximate of the form uy(#) = wi (yu). We provide a brief 
overview of methods developed to address the challenge. We restrict our coverage 
to methods tested for multidimensional aerodynamics problems, and refer to the 
references in [53] and a review paper [54] for a more general coverage. 

Domain decomposition. One way to address the problem is to forgo the reduction 
of the state over the entire domain and to only reduce solution over a portion of the 
domain, as proposed for transonic Euler flows by LeGresley and Alonso [45]. Namely, 
we first decompose the domain into two regions: (i) region Q om ¢ Q over which the 
solution varies smoothly and hence {u,(}) lo, Juep is amenable to model reduction, 
and (ii) region Qfom = Q \ Qrom Which contains moving discontinuities and hence is 
not amenable to model reduction. We then approximate the solution up(W)|o, using 
a reduced basis {¢ j loron Yea and u, (Ņ)| es, using the native basis of the FOM. 
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Nonlinear model reduction. Another approach to address moving discontinuities 
is to consider nonlinear model reduction. Here, nonlinear model reduction refers to 
approaches that approximate the solution in not a linear space Vy but in a nonlinear 
space. (Nonlinear model reduction should not be confused with linear model reduc- 
tion of nonlinear PDEs, which has been considered so far in this section.) Nonlinear 
model reduction approaches considered by both Cagniart et al. [19] and Nair and Bal- 
ajewicz [53] are based on the following observation: If the snapshots can be trans- 
lated in space such that the shocks are aligned, then the snapshots can be effectively 
compressed using a linear model reduction technique (e. g., POD). Specifically, the 
approach approximates the solution uņp(; 4) € Vp by 


Uy (XM) = 7 Os pu p) 
for some uy (fl) € RY and a parameter-dependent basis 
Gm) =u uu) j=1...N, 


where yj : Qx P > RÍ, j=1,...,N, are parameter-dependent translation functions. 
The translation functions {y;} are trained in the offline stage such that the shock lo- 
cations for the translated basis Q (; u) = Unlyj( P)) are (approximately) aligned with 
the shock in u,,(-; 4). Nonlinear approximation of shocks is a relatively new develop- 
ment in the field of model reduction, and hence we refer to [19, 53, 54] and references 
therein for specific implementations. The nonlinear model reduction approach has 
been applied to transonic Euler over an airfoil [19] and supersonic forward step [53]. 


6.4.5 Large-scale applications 


We conclude this section with a few examples of model reduction applied to large-, 

industry-scale nonlinear aerodynamics problems. 

— Unsteady turbulent flow past Amhed body [22]. In this work Carlberg etal. con- 
sider model reduction of nonparameterized turbulent flow over the Ahmed body 
modeled by detached eddy simulation. The FOM consists of N, ~ 1.7 x 10’ spatial 
degrees of freedom. The FOM is hyperreduced using the GNAT method; the result- 
ing ROM uses a reduced basis of the size N = 283 for the state, a reduced basis of 
the size Np = N; = 1,514 for the residual and Jacobian, and N, = 378 sample nodes. 
The ROM reproduces the unsteady drag time history with less than 1% discrep- 
ancy. The FOM requires 13 hours using 512 cores, whereas the ROM requires 3.9 
hours using 4 cores; the ROM reduces the computational cost by a factor of 438. 

- Parametric shape deformation of the NASA Common Research Model [69]. In this 
work Washabaugh et al. consider model reduction of steady RANS-SA flow over 
the NASA Common Research Model under parametric shape deformation. The 
FOM consists of N, = 6.8 x 10’ degrees of freedom and is parameterized by four 
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shape parameters: wingspan, washout, streamwise wingtip rake, and vertical 
wingtip rake. The ROM based on the minimum-residual formulation with the 
gappy POD collocation hyperreduction uses N = 23 modes and N, = 5000 sample 
nodes. The ROM achieves less than 0.3 % error in drag for test parameters consid- 
ered. A single simulation of the FOM requires 2 hours using 1024 cores, whereas 
the ROM requires 2.8 minutes on a laptop. 


6.5 Summary and conclusions 


In this chapter, we surveyed model reduction techniques for linearized and nonlin- 
ear aerodynamics problems that have been developed in the past two decades. We 
discussed essential ingredients of model reduction, with an emphasis on techniques 
that are designed to address challenges in aerodynamics, including convection dom- 
inance, nonlinearity, limited stability, limited regularity, and a wide range of scales. 
We also reviewed successful applications of model reduction to large-scale industry- 
relevant aerodynamics problems to date. There still exist many open challenges to 
model reduction of complex aerodynamics problems. Their industrial relevance and 
challenging nature make them arguably an ideal testbed to develop and assess the 
next generation of model reduction algorithms. 
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Bülent Karas6zen 
7 Model order reduction in neuroscience 


Abstract: The human brain contains approximately 10° neurons, each with approxi- 
mately 10° connections, synapses, with other neurons. Most sensory, cognitive, and 
motor functions of our brains depend on the interaction of a large population of neu- 
rons. In recent years, many technologies have been developed for recording large 
numbers of neurons either sequentially or simultaneously. Increases in computa- 
tional power and algorithmic developments have enabled advanced analyses of the 
neuronal population parallel to the rapid growth of quantity and complexity of the 
recorded neuronal activity. Recent studies made use of dimensionality and model 
order reduction techniques to extract coherent features which are not apparent at the 
level of individual neurons. It has been observed that the neuronal activity evolves 
on low-dimensional subspaces. The aim of model reduction of large-scale neuronal 
networks is the accurate and fast prediction of patterns and their propagation in dif- 
ferent areas of the brain. Spatiotemporal features of brain activity are identified on 
low-dimensional subspaces with methods such as dynamic mode decomposition, 
proper orthogonal decomposition, the discrete empirical interpolation method, and 
combined parameter and state reduction. In this chapter, we give an overview of the 
currently used dimensionality reduction and model order reduction techniques in 
neuroscience. 


Keywords: neuroscience, dimensionality reduction, proper orthogonal decomposi- 
tion, discrete empirical interpolation, dynamic mode decomposition, state and pa- 
rameter estimation 


MSC 2010: 93A15, 92C55, 37M10, 37M99, 37N40, 65R32 


7.1 Introduction 


Due to the advances in recording and imaging technologies, the number of recorded 
signals from brain cells increased significantly in the last few years. The recorded 
spatio-temporal neural activity gives rise to networks with complex dynamics. In neu- 
roscience, molecular and cellular level details are incorporated in large-scale models 
of the brain in order to reproduce phenomena such as learning and behavior. The rapid 
growth of simultaneous neuronal recordings in scale and resolution brings challenges 
with the analysis of the neuronal population activity. New computational approaches 
have to be developed to analyze, visualize, and understand large-scale recordings of 
neural activity. While algorithmic developments and the availability of significantly 
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more computing power have enabled analysis of larger neuronal networks, these ad- 
vances cannot keep pace with the increasing size and complexity of recorded activity. 
The activity of complex networks of neurons can often be described by relatively few 
distinct patterns. Model order reduction techniques enable us to identify the coherent 
spatio-temporal patterns. 

The presence or absence of a neural mechanism can be analyzed for neuronal 
populations. Dimensionality reduction methods [6] are data-driven statistical tech- 
niques for forming and evaluating hypotheses about population activity structure, 
which are summarized in Section 7.2. One of the goals of neuroscience is the fast and 
accurate prediction of the potential propagation in neurons. The differential equa- 
tions describing the propagation of potential in neurons have been developed and 
are described by Hodgkin and Huxley equations [12]. They consist of a coupled sys- 
tem of ordinary and partial differential equations (ODEs and PDEs). The dimension 
of the associated discretized systems is very large for accurately simulating neurons 
with realistic morphological structure and synaptic inputs. In Section 73 we present 
two model order reduction approaches based on proper orthogonal decomposition 
(POD) and the discrete empirical interpolation method (DEIM) [5], which can predict 
accurately the potential propagation in large-scale neuronal networks leading to im- 
portant speedups [17, 16, 2]. Using the functional neuroimaging data from electroen- 
cephalography (EEG) or functional magnetic resonance imaging (fMRI), different re- 
gions of the brain can be inferred by dynamic causal modeling (DCM) [7]. Effective 
connectivity is parameterized in terms of coupling among unobserved brain states 
and neuronal activity in different regions. In Section 7.4 we describe a combined state 
and parameter reduction for parameter estimation and identification [10] to extract 
effective connectivity in neuronal networks from measured data, such as EEG or fMRI 
data. In Section 7.5 the data-driven, equation-free model order reduction method dy- 
namic mode decomposition (DMD) is described for identifying sleep spindle networks 
[3]. Reduced-order models (ROMs) with POD and the DEIM and four variants of them 
are presented for neuronal synaptic plasticity and neuronal spiking networks in Sec- 
tion 7.6. 


7.2 Dimensionality reduction methods 


Coordination of responses across neurons exists only at the level of the population 
and not at the level of single neurons. The presence or absence of a neural mecha- 
nism can be analyzed for neuronal populations. Dimensionality reduction methods 
are data-driven statistical techniques for forming and evaluating hypotheses about 
population activity structure. They produce low-dimensional representations of high- 
dimensional data with the aim to extract coherent patterns which preserve or highlight 
some feature of interest in the data [6]. The recorded neurons of dimension D are likely 
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not independent of each other, because they belong to a common network of neuronal 
populations. From the high-dimensional data of neuronal recordings, a smaller num- 
ber of explanatory variables K (K < D) are extracted with the help of dimensional- 
ity reduction methods. The explanatory variables are not directly observed; therefore 
they are referred to as latent variables. The latent variables define a K-dimensional 
space representing coherent patterns of the noisy neural activity of D neurons. 

There exist several dimensionality reduction methods which differ in the statisti- 
calinterpretation of the preserved and discarded features of the neuronal populations. 
We summarize the commonly used statistical methods of dimensionality reduction 
methods in [6], where further references about the methods can be found. 

Principal component and factor analysis. The most widely known technique to ex- 
tract coherent patterns from high-dimensional data is modal decomposition. A par- 
ticularly popular modal decomposition technique is principal component analysis 
(PCA), which derives modes ordered by their ability to account for energy or variance 
in the data. In particular, PCA is a static technique and does not model temporal dy- 
namics of time-series data explicitly, so it often performs poorly in reproducing dy- 
namic data, such as recordings of neural activity. The low-dimensional space identi- 
fied by PCA captures variance of all types, including firing rate variability and spiking 
variability, whereas factor analysis discards the independent variance for each neu- 
ron. and preserves variance that is shared across neurons. 

Time series methods. The temporal dynamics of the population activity can be 
identified if the data come from a time series. The most commonly used time series 
methods for dimensionality reduction neural recordings are hidden Markov models 
(HMMs) [18], kernel smoothing followed by a static dimensionality reduction, Gaus- 
sian process factor analysis [35], latent linear dynamical systems [4], and latent non- 
linear dynamical systems [26]. They produce latent neural trajectories that capture the 
shared variability across neurons. The HMM is applied when a jump between discrete 
states of neurons exists, other methods identify smooth changes in firing rates over 
time. 

Methods with dependent variables. On many neuronal recordings the high-dimen- 
sional firing rate space is associated with labels of one or more dependent variables, 
like stimulus identity, decision identity, or a time index. The dimensionality reduction 
aims in this case to project the data such that differences in these dependent variables 
are preserved. Linear discriminant analysis can be used to find a low-dimensional pro- 
jection in which the G groups to which the data points belong are well separated. 

Nonlinear dimensionality reduction methods. All the previous methods assume a 
linear relationship between the latent and observed variables. When the data lie ona 
low-dimensional, nonlinear manifold in the high-dimensional space, a linear method 
may require more latent variables than the number of true dimensions of the data. 
The most frequently used methods to identify nonlinear manifolds are Isomap79 [31] 
and locally linear embedding [28]. Because the nonlinear methods use local neighbor- 
hoods to estimate the structure of the manifold, population responses may not evenly 
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explore the high-dimensional space. Therefore theses methods should be used with 
care. 


7.3 Proper orthogonal decomposition and discrete 
empirical interpolation for the Hodgkin—Huxley 
model 


One of the goals of neuroscience is the fast and accurate prediction of the potential 
propagation in neurons. The differential equations describing propagation of poten- 
tial in neurons are described by Hodgkin—Huxley (HH) cable equations [12]. They con- 
sist of a coupled system of ODEs and PDEs. Accurate simulation of morphology, ki- 
netics, and synaptic inputs of neurons requires solution of large systems of nonlin- 
ear ODEs. The complexity of the models is determined by the synapse density and 
the dentritic length. In simulations, for one synapse per micron on a cell with a den- 
drite of 5mm, 5,000 compartments each with 10 variables are needed, which results 
in 50,000 coupled nonlinear system of ODEs [17, 16]. To recover complex dynamics, ef- 
ficient reduced-order neuronal methods are developed using POD and the DEIM from 
the snapshots of the in space and time discretized coupled PDEs and ODEs [17, 16, 2]. 
In this section we describe two of them. They differ in the formulation of the HH cable 
equation and of the equations for the gating variables. 


7.3.1 Morphologically accurate reduced-order modeling 


The neuronal full-order models (FOMs) in [17, 16] consist of D branched dendritic neu- 
rons B = eae Bq meeting at the soma, where the d-th neuron has B4 branches. It is 
assumed that the branch b carries C distinct ionic currents with associated densities 
and G,,(x) and reversal potentials E., c = 1,...,C. The kinetics of current c on branch 
b are governed by the F, gating variables, w,,, f = 1,...,F,. When subjected to in- 
put at S, synapses, the nonlinear HH cable equation for the transmembrane potential 
vp(x, t) with the equation for the gating variables wpe is given by (see [2], Figure 1, 
model network with three cables) 


OVp _ 1 oO 3c 
pC at sel box 


c F, 
= Ay È GrclX)Vp - Ee) | [ wy 
c=1 fel 


Sp 
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OWycf  Wef,oo (Vb) — Whef 
ot Tof(Vp) 


, O<x<l,, t>0, (7.2) 


where g;,,(nS) is the time course, Xp; is the spatial location, and E}; is the reversal 
potential of the s-th synapse on branch b. The variables and parameters in (7.1) are 
described in [17, 16]. 

These branch potentials interact at J junction points, where junction J denotes the 
soma. The D dendrites join at the soma. Continuity of the potential at the soma leads 
to a common value at the current balance denoted by v,(t). Then the networked form 
of (7.1) becomes 


oi d=1 
c 


F, 
— ap $ Goc(X)(Vo i Eo) [wO 
c=1 f=1 


Sp 
+ È Sos(t)(Vo(t) = Egs)» (7.3) 
0 s=1 
OWocf (Et) E Wef,œo(Vo(t)) — Wocf (t) 
FT CAG) > O<x<l, t>0. (7.4) 


When the cell is partitioned into N compartments, with C distinct ionic currents 
per compartment and with F gating variables per current, the following nonlinear 
ODEs are obtained: 


v' (t) = Hv(t) - (®(w(t))e).v(t) + D(w(t))E; 


+ G(t).(v(t)-E,), v(t) € RY, (7.5) 
w' (t) = (A(v(t)) - w(t))./B(v(t)), wE) e ROF, (7.6) 
where H € RY" is the Hines matrix [11], e = [1 1---1]’ € RE and the “dot” operator, 


a.b, denotes elementwise multiplication; E; and E, are the vector of channel reversal 
potentials and the vector of synaptic reversal potentials, respectively. Equation (7.5) is 
discretized in time by the second-order discretized Euler scheme [11]. 

In [16] using the snapshots of v(t) and of the nonlinear term N(v(t),w(t)) = 
(M(w(t))e).v(t) — B(w(t))E; at times t,,t),...,t, the POD and DEIM modes are con- 
structed. 

The reduced membrane potentials v, are constructed using the POD basis, and 
the reduced gating variables w, are obtained after applying the DEIM to the nonlinear 
terms. The ROM in [16] preserves the spatial precision of synaptic input and captures 
accurately the subthreshold and spiking behaviors. 

In [17] a linearized quasi-active reduced neuronal model is constructed using bal- 
anced truncation and ,-approximation of transfer functions in time. ROMs preserve 
the input-output relationship and reproduce only subthreshold dynamics. 


242 — B.Karasézen 


7.3.2 Energy-stable neuronal reduced-order modeling 


In [1, 2] a different form of the HH cable equation and ODEs for gating variables is con- 
sidered. The intracellular potential v(x, t) and three gating variables m(x, t), h(x, t), 
and n(x, t) describe the activation and deactivation of the ion channels, of the sodium 
channels, and of the potassium channels, respectively. A single cable in the computa- 
tional domain (x, t) € [0, L] x (0, T] describing the distribution of the potential u(x, t) 
is given by [1, 2] 


ou 1 1 
TE aay (a) Ua - g 8m hy nut f(m h,n,x t), (7.7) 
m m 
where u = € x > 0, a(x) is the radius of the neurons, C,,, is the specific membrane 


capacitance, and R; is the axial resistivity. The conductance g(x, t) is a polynomial of 
the gating variables 


g(x, t) = gmh + gn’ +8 > 0, (7.8) 
with the source term 
f(m,h,n,x, t) = g)E,m*h + gEn + 93E; — i(x, t), (79) 


where E), l= 1,2, 3, are equilibrium potentials and i(x, t) is the input current at x, 
N, 
iœt) =J it), x € [0,L]. (7.10) 


s=1 


The nonlinear ODEs for the gating variables are given by 


om = a, (v(x, t)(1 - m(x, t)) - Bwv (x, t))m(x, t), 
a = a, (v(x, (1 - h(x, t) - B,v(x, Dh, t), (7:1) 
0 


n 
ee a, (v(x, t))(1 — n(x, t)) - Buv (x, t))n(x, t). 
Expressions for Qn» &h, An» m» Bh» By and boundary conditions can be found in [2]. 

In [1, 2], a model network with three cables connected to a soma is used. The equa- 
tions governing the potential propagation in a network of N, neuron cables-dendrites 
and/or axons with the superscript © a N,, are given as 


av H ORONA) 1 (c) p(0) (0), (0) 
at = Foy l (x) Vx z gem Wen )u 


1 
EE (mO, hO, nO, x,t), 712) 
m 
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© 
oi = Am (vO (1 - m) - Bav)m, 
ah 
T anl(vO)(1 - h®) - Bav(c))h®, (713) 
© 
T = ay(v))(1 n) = Byv yn, 


for x € A = [0, L] together with the boundary conditions. 

The semi-discrete forms of these equations are approximated using energy-stable 
summation by parts [1, 2] for the model network. The reduced-order bases (ROBs) for 
multiple cables of identical lengths are assembled into a network in block form. The 
block structure of the ROB allows a flexible structure-preserving model reduction ap- 
proach with an independent approximation in each cable and energy stability and 
accuracy properties follow from this block structure. Computation of the time-varying 
reduced variables in the gating equations at every time t is costly because they scale 
with the dimension of the FOM. A nonnegative variant of the DEIM, nonnegative DEIM 
(NNDEIM), is developed in [2] to preserve the structure and energy stability properties 
of the equations. 

The capability of the greedy-based approach to generate accurate predictions in 
large-scale neuronal networks is demonstrated for systems with more than 15,000 de- 
grees of freedom. The state variable ROB of dimension l = 15 with POD modes and the 
nonnegative ROBs of dimension p = 60 with NNDEIM modes are constructed using a 
greedy approach to predict the potential variation at the soma. The speedup factor of 
simulations is about 20, which is larger than that of Galerkin projection, which is only 
1.3 without the NNDEIM. 


7.4 Combined state and parameter reduction for 
dynamic causal modeling 


In neuroscience, different regions of the brain are inferred using neuroimaging data 
from EEG or fMRI recordings using the method of DCM [7]. Effective connectivity is 
parameterized in terms of coupling among unobserved brain states and neuronal ac- 
tivity in different regions. In DCM the neuronal activity of the observed brain region is 
represented as a single-input single-output (SISO) system 


X = Agyn(W)x + Baynus (7.14) 


with the parameterized connectivity A gy,(#) and external input matrices Bayn- 
Linearization of the nonlinear DCM hemodynamic forward submodel (balloon 

model) [7] transforms the neuronal activity to the measured blood oxygen level- 

dependent (BOLD) response. Linearization around the equilibrium results in the 
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following SISO system: 
Bops = 1000)", Cops = (0 0 Vok, Voko), (7.15) 
Z = AobsZi + BonsXip (7.16) 
Yi = ConsZi» (7.17) 
Zo = (0000)', (718) 
1 1 
7 an 
"N 1 0 0 o0 (719) 
ae o ṣg0-0-E)0-In(1-E)) 5 E$ i 
a 1 
(0) To 0 Toa 


The fMRI measurements at the i-th brain region are reflected by the output vari- 
ables y;. For the meaning of the variables and parameters in (7.15) and (7.19) we refer 
to [10, 9]. The linearized forward submodels are embedded into the fMRI connectivity 
model 


x Agyn(h) 0 0) -- 0 x 
zy 61 Aobs 0 24 
Zo = 654 O Aos Z2 


ZN a ÔL Nan Aobs ZNayn 


B dyn 


a 0 i (7.20) 


N 
= 


yali 7 oe pik (7.21) 


Cobs 
ZN, dyn 


where ô;j € R Nam denotes the Kronecker matrix with nonzero elements located at 
the (i, j)-th component. 

The linearized state-space forward model (7.20) and (7.21) corresponds to a mult- 
iple-input multiple-output (MIMO) system, 


X(t) = A(u)x(t) + Bu(t), y(t) = Cx(t), (7.22) 
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where x € R” is the internal state, u € R’ is the external input, y € R° is the observed 
output, and p are the parameters describing different conditions. 

For large numbers of M := N? parameters, the computational cost for inferring the 
parameters and states is very high. In [10, 8] a combined state and parameter model 
order reduction is developed for parameter estimation and identification to extract ef- 
fective connectivity. The inversion procedure consists of two phases, the offline and 
online phases. In the offline phase, the underlying parameterized model is reduced 
jointly in states and parameters. In the online phase, the ROM’s parameters are es- 
timated to fit the observed experimental data. Using state and parameter reduction, 
the computational cost is reduced in the offline phase. The simultaneous reduction of 
state and parameter space is based on Galerkin projections with the orthogonal ma- 
trices for the state V € IR“*" and for the parameters P € R, The reduced model is 
of lower order n « N,m « M than the original FOM. The reduced states x,(t) € R” 
and the reduced parameters u € IR” are computed as 


X,(t) = A,(u,)x,(t) + B,ult), y,(t) = C,x(t), (7.23) 
with a reduced initial condition x, 9 = V"x, and the reduced components 


u, =P’ € R”, 

A,(u,) = VTA(Pu,) V € R™", 
B,=V'BeR™, 
C, = CV € RO", 


In the online phase, the optimization-based inverse problem is combined with the 
reduction of state and parameter space. The inversion is based on a generalized data- 
driven optimization approach to construct the ROMs in [23] and enhanced with the 
Monte Carlo method to speed up the computations. The state projection V € R” and 
parameter projection P € R™*” are determined iteratively based ona greedy algorithm 
by maximizing the error between the high-fidelity original and the low-dimensional 
reduced model in the Bayesian setting. 

Numerical experiments with the DCM model in [23] show the highly dimensional 
neuronal network system is efficiently inverted due to the short offline durations. In 
the offline phase, Monte Carlo-enhanced methods require more than an order of mag- 
nitude less offline time compared to the original and data misfit-enhanced methods. 
In the online phase the ROM has a speedup factor of about an order of magnitude more 
compared to the full-order inversion. The output error of the data misfit-enhanced 
method is close to that of the full-order method. The output errors of Monte Carlo de- 
crease with increasing numbers of simulation but do not reach the output error of the 
full-order system. The source code is available in MATLAB [8]. 
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7.5 Dynamic mode decomposition 


DMD is a data-driven, equation-free ROM technique [20]. It was initially developed 
to reduce the high-dimensional dynamic data obtained from experiments and sim- 
ulations in fluid mechanics into a small number of coupled spatio-temporal modes 
[29, 30]. DMD was applied to explore spatio-temporal patterns in large-scale neuronal 
recordings in [3]. DMD can be interpreted as combination of discrete Fourier transform 
(DFT) in time and PCA [14] in space. Both PCA and independent component analysis 
[13] are static techniques, which perform poorly in reproducing dynamic data, such as 
recordings of neural activity. 

The data are taken from electrocorticography (ECoG) recordings. Voltages from 
n channels of an electrode array were sampled every At. These measurements are ar- 
ranged at snapshot k to the column vectors x,. The m snapshots in time construct data 
matrices shifted in time with At, 


| | | | | | 
X=| X% X% ce Xma b X=| X% X S| (7.24) 


These matrices are assumed to be related linearly in time, 
X' = AX. (7.25) 


The DMD of the data matrix pair X and X’ is given by the eigendecomposition of A 
using the singular value decomposition of the data matrix X = UZXV* by computing 
the pseudo-inverse A = X'X' = X'VE tU”. The spatio-temporal modes are computed 
by the exact DMD algorithm [32]. 

Because DMD does not contain explicit spatial relationships between neighboring 
measurements, traveling waves occurring in neuronal networks cannot be captured 
well with a few coherent modes. DMD was also used as a windowed technique with a 
temporal window size constrained by lower bound as for DFT. In contrast to the fluid 
dynamics where n > m, in neuroscience the electrode arrays have tens of channels 
nin the recordings with m number of snapshots in the windows data per second, so 
that n < m. The number of singular values v of X are less than n and m — 1, which 
restricts the maximum number of DMD modes and eigenvalues to n. Because of this 
the dynamics can be captured over m snapshots. The rank mismatch is resolved by 
appending to the snapshot measurements with h — 1 time-shifted versions of the data 
matrices. The augmented data matrix Xau¢ is given as 
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XX) Xm-h 
X% ë X Xm-h-1 

Xaug = (7.26) 
Xh Xhit Xm-1 


The augmented matrices Xj, and Kg are Hankel matrices, which are constant 


along the skew diagonal, as in the eigenvalue realization algorithm [15]. The number 
of stacks his chosen such that hn > 2m. A measure to determined the optimal number 
of stacks h is the approximation error 


g- WRK 
[X] 


where || - ||p is the Frobenius norm. The approximation error E is decreasing with in- 
creasing number of stacks h and reaches a plateau, so that the DMD accuracy does not 
significantly increase. 

DMDis applied in [3] as an automated approach to detect and analyze reliably spa- 
tial localization and frequencies of sleep spindle networks from human ECoG record- 
ings. A MATLAB implementation is available at github.com/bwbrunton/dmd-neuro/. 


7.6 Reduced-order modeling of biophysical neuronal 
networks 


Recently ROMs for ODEs 
X(t) = A(f)x(t) + f (x(t)) + Bu(t) (7.27) 


were constructed using POD and the DEIM to investigate input-output behavior of the 
neuronal networks in the brain [22, 21], where x(t) are state and u(t) are input vari- 
ables. 

The model in [22] is based on the chemical reactions of molecules in synapses, that 
are the intercellular information transfer points of neurons. The signaling pathways 
in striatal synaptic plasticity are modeled in [19]. This model describes how certain 
molecules, which are a prerequisite for learning in the brain, act in synapses. The 
stoichiometric equations obey the law of mass action, which leads to a deterministic 
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system of 44 ODEs of the form (7.27). The state x(t) of the control system (7.27) is a 
collection of ions, molecules, and proteins that act in neuronal synapses. The linear 
part of (7.27) is sparse, the nonlinearities are quadratic. The time-dependent stimulus 
u(t) consists of molecules that are important for neuronal excitability and plasticity, 
calcium and glutamate. 

In [21], a nonlinear biophysical network model is considered, describing synchro- 
nized population bursting behavior of heterogeneous pyramidal neurons in the brain 
[27]. Neurons communicate by changing their membrane voltage to create action po- 
tentials (spikes), propagating from cell to cell. Spiking is the fundamental method of 
sensory information processing in the brain, and synchronized spiking is an emergent 
property of biological neuronal networks. The ODE system (7.27) in [21] consists of the 
states x(t) as a collection of 50 neurons, each modeled with 10 ODEs, leading totally 
to a system of ODEs of dimension 500. Each cell is modeled with HH equations, where 
each cell has only two compartments (dendrites and soma) and these compartments 
have different ion channels. The state variables x(t) include the voltages of somatic 
and dendritic compartments, the dendritic calcium concentration, and synaptic and 
ion channel gating variables, and the input u(t) is an injected current. Additionally, 
the soma compartment voltages are coupled to dentritic compartments of randomly 
chosen cells. This networking of the output of cells as input to other cells is key for 
producing synchronized population behavior. The nonlinearities are HH type, i.e., 
exponential functions as well as cubic and quartic polynomials. 

In [22], POD-DEIM was applied to a data-driven biological model of plasticity in 
the brain (7.27). The ROMs with POD-DEIM reduce significantly the simulation time 
and error between the original model and reduced-order solutions can be tuned by ad- 
justing the number of POD and DEIM bases independently. When the ROMs are trained 
in a matching time interval of 10,000 seconds, accurate results are obtained. However, 
generalizing the reduced model to longer time intervals is challenging, which is char- 
acteristic for all nonlinear models. 

In [21], the network model (7.27) is reduced with localized DEIM [24], discrete adap- 
tive POD (DAPOD) [33, 34], and adaptive DEIM (ADEIM) [25]. DEIM and the variations 
are used here in combination with POD. ROMs require large numbers of POD and DEIM 
bases, to accurately simulate the input-output behavior in the ROMs. In this model, 
every cell is heterogeneous in parameters and there are also jump/reset conditions, 
which are factors that pose additional challenges to the ROMs. However, the ROMs in 
[21] were able to replicate the emergent synchronized population activity in the origi- 
nal network model. DAPOD and ADEIM perform best in preserving the spiking activity 
of the original network model. ADEIM is too slow and does not allow low enough di- 
mensions to offset the computational costs of online adaptivity. DAPOD is able to find 
a lower-dimensional POD basis online than the other methods find offline, but the 
runtime is close to that of the original model. 
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Niccolo Dal Santo, Andrea Manzoni, Stefano Pagani, and 
Alfio Quarteroni 


8 Reduced-order modeling for applications 
to the cardiovascular system 


Abstract: The capability to provide fast and reliable numerical simulations is of 
paramount importance when dealing with complex applications arising from medi- 
cine. More than for other branches of engineering and applied sciences, performing 
accurate computations in a short amount of time — minutes, rather than hours, or even 
days — is crucial when dealing with problems arising from life sciences, like, e. g., in 
the simulation of the cardiovascular system. Moreover, many sources of variability 
carried by subject-specific features have to be incorporated into the mathematical 
models, to quantify their impact on the computed results. For these reasons, bringing 
computational results into clinical practice represents a great challenge. Reduced- 
order modeling techniques such as the reduced basis method represent a key tool 
towards the possibility to address these challenges, thus making the numerical mod- 
eling of the cardiovascular system a new, fascinating testbed for these methodologies. 


Keywords: reduced basis method, proper orthogonal decomposition, hyperreduction 
techniques, hemodynamics, cardiac electrophysiology 
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8.1 Numerical simulations in clinical practice 


The numerical modeling of the cardiovascular system is a research topic that has at- 
tracted remarkable interest from the scientific computing community because of the 
intrinsic mathematical and computational difficulty, and due to the increasing impact 
of cardiovascular diseases worldwide. A wealth of models are nowadays available to 
address both physiological and pathological instances, aiming at better understand- 
ing the quantitative processes governing the blood circulation and opening new fron- 
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tiers in therapeutic planning and design of implantable devices (e. g., medical stents 
and cardiac defibrillators). Numerical simulations provide useful insights into the be- 
havior of quantities which cannot be measured directly, such as the wall shear stress 
(WSS) over a portion of the lumen boundary or the transmembrane potential in the 
myocardium. This is meant to enable quantitative analysis in several virtual scenarios 
to support clinicians’ decisions and to enhance common diagnostic practices based 
on medical imaging. 

However, a number of difficulties (including, e. g., the lack of several physical pa- 
rameters or the uncertainty affecting their values, as well as the presence of a wide 
range of spatio-temporal scales) arises when solving problems occurring in the de- 
scription of the cardiovascular system. Even more importantly, their numerical simu- 
lation by means of high-fidelity approximation techniques, also called full-order mod- 
els (FOMs), such as the finite element (FE) method might be extremely demanding. 
This is the case, for instance, of numerical simulations of cardiac electromechanics, 
as well as the description of fluid dynamics of blood flowing through the heart cham- 
bers (two ventricles and two atria). The numerical solution of these problems - ex- 
pressed in the form of systems of partial differential equations (PDEs) - easily in- 
volves up to 0(10°) degrees of freedom and several hours (or even days) of compu- 
tational time, also on powerful parallel architectures. Consequently, high-fidelity ap- 
proximation techniques become prohibitive when we expect them to deal quickly, 
but accurately, with the repetitive solution of problems in view, e.g., of model per- 
sonalization — that is, the adaptation of model inputs to subject-specific conditions. 
For instance, when simulating the electric activity of the heart (Section 8.2.2) the in- 
puts to be personalized may include the shape of the domain (e. g., the geometry of 
atria and ventricles), physical parameters (e. g., electric conductivities or the orien- 
tation of tissue fibers), and initial and boundary conditions. In this context, physical 
indices and outputs of clinical interest can be directly approximated through the nu- 
merical solution of nonlinear parameterized coupled systems of PDEs, such as the 
bidomain or monodomain equations, equipped with a system of ordinary differen- 
tial equations (ODEs) encoding suitable ionic models; see, e. g., [20, 51, 53]. Further- 
more, very often input/output evaluations represent building blocks of more complex 
problems, as data assimilation, parameter estimation, and uncertainty quantification 
problems. 

To reduce the computational complexity of the FOM in all these contexts, reduced- 
order models (ROMs) such as the reduced basis (RB) method for parameterized PDEs 
represent efficient techniques for the approximation of the parameterized PDE solu- 
tion. Although several works have focused on problems related to both hemodynamics 
[5, 6, 43] and the simulation of cardiac function [12, 11, 21, 27, 41, 47], applying state- 
of-the-art ROMs is not straightforward for cardiac problems because of (i) nonlinear 
behavior (like sharp moving fronts in the case of cardiac electrophysiology) and (ii) 
parameterization of complex geometries. For these reasons, suitable strategies must 
be devised to build low-dimensional RB spaces able to capture the manifold of the 
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problem solutions when varying the parameters, by keeping the cost of the ROM con- 
struction sufficiently low. Local (or nonlinear) techniques to build RB spaces and effi- 
cient, purely algebraic hyperreduction strategies can help. 

In this chapter we review the current state-of-the-art construction of efficient and 
accurate RB methods for the solution of parameterized problems dealing with (i) arte- 
rial fluid dynamics and (ii) the electric activity of the heart. Throughout the chapter, 
H € P denotes a parameter vector entering in the definition of the PDE model, whose 
components might represent physical and/or geometrical features of interest; P c RP 
denotes the parameter space. 

The structure of this chapter is as follows. In Section 8.2 some insights on the two 
problems we focus on — namely, arterial blood flow and cardiac electrophysiology — 
are provided. In Section 8.3 we sketch the main difficulties arising when dealing with 
the construction of ROMs for these two problems, addressing some possible strategies 
to enhance computational efficiency and numerical accuracy of ROMs. Numerical re- 
sults related to arterial blood flow and cardiac electrophysiology are presented in Sec- 
tions 8.4 and 8.5, respectively. Finally, open critical issues and future perspectives are 
briefly outlined in Section 8.6. For the sake of space, we do not report the full descrip- 
tion of the FOMs and the ROMs involved in the solution of the proposed problems; the 
interested reader can find further details in [25, 47]. 


8.2 Two relevant cardiovascular applications 


8.2.1 Arterial blood flow 


The cardiovascular system is a closed circuit that carries oxygenated blood to all the 
tissues and organs of the body. The systemic circulation is made up of the arteries, 
carrying oxygenated blood ejected by the left heart to the living tissues, and the veins, 
allowing nonoxygenated blood to return to the right heart. In large arteries, blood be- 
haves like a Newtonian fluid, modeled by unsteady Navier-Stokes equations, with pul- 
satile input. The different velocity of blood flow in the arteries of the systemic circu- 
lation results in different values of the Reynolds number Re = pDU/p (where D and U 
are characteristic vessel dimension and blood velocity, respectively), a dimensionless 
quantity which highlights the importance of the inertial terms over the viscous ones. 
Here we assume to deal with laminar flows. Indeed, blood experiences a wide range 
of Reynolds numbers; moreover, the pulsatile nature of blood flow does not allow the 
onset of fully turbulent flow in healthy conditions. This is not necessarily the case for 
some pathological conditions, such as (severe) carotid stenosis, yielding a narrowing 
of the vessel lumen and increased complexity of the geometry together with higher 
Reynolds numbers [38]. Occlusions (or stenoses) at the carotid bifurcation are caused 
by the accumulation of fatty material in the internal layer of the vessel wall, progres- 
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sively leading to plaque formation and atherosclerosis. The main complications are 
partial occlusion of the lumen with possible generation of cerebral ischemia, or even 
total occlusion, resulting in cerebral infarction. The role of blood fluid dynamics has 
been recognized as crucial for the development of such a disease [67, 39, 63]. In par- 
ticular, WSSs, that is, the viscous/friction forces exerted by the blood on the vessel 
wall, despite being 100 times smaller in magnitude than pressure, play an important 
role in atherosclerosis. For the case at hand, we will consider less severe occlusions 
occurring at the carotid bifurcation, thus justifying the assumption of laminar flow. 

In particular, we are interested to characterize blood flow patterns in different 
geometrical configurations obtained by increasing the degree of stenosis of a subject- 
specific carotid bifurcation geometry, for a wide range of flow conditions. Hence, both 
geometrical and physical parameters are considered: The former are related with the 
computational domain, the latter with the problem’s data (affecting the Reynolds 
number). Given an open bounded and p-dependent domain Q(u) c RÎ, d = 2,3, 
such that Q(U) = Tour(H) UT in(H) UTD and Pour) N Éin) = PO) 9 Vin) = 
Tout) N Ty (H) = 9, and a final time T > 0, unsteady Navier-Stokes equations for 
Newtonian, incompressible fluids read as follows: For each t € (0, T), 


EW qu) -VAD - V- oH). PQD) +YP =Ö in OG, 

V -ui(u) = 0 in Q(p), 

ii(u) = 0 on T„ (4), (8.1) 
üu) = Bys(H) on Tin (H), 
oü), pW ÄH) = Ö on Tout (H), 

ù(u) = üo in Q(y), att = 0. 


Here ù(u) and p(y) are the velocity and the pressure of the fluid; o(u(y),p(w)) = 
—p(W)I + 2ve(ù(u)) denotes the stress tensor; and v = p/p denotes the kinematic vis- 
cosity of the fluid, p being the blood density and v its viscosity. The strain tensor is 
given by e(ù(u)) = SV) + Vii(u)"). Note that also the Dirichlet boundary condition 
Sys(H) might depend on u. We avoid to deal with fluid—structure interaction (FSI) 
for the problem at hand; indeed, arterial vessels are compliant, yielding quite large 
wall displacements (reaching up to 10% of the lumen diameter). The reduction of 
FSI problems has only been partially addressed in few works, focusing on simplified 
geometries, and without addressing efficient hyperreduction techniques; see, e.g., 
[37, 8]. 

Problem (8.1) is first discretized in space by means of the FE method, relying on 
quadratic and linear FEs for velocity and pressure, respectively, and then in time with 
a semi-implicit backward differentiation formula (BDF) scheme of order ø = 2. We 
introduce a partition of [0, T] in N, subintervals of equal size At = T/N,, such that 
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t, = nAt, and approximate the time derivative of the FOM velocity ui, (y) at t, as 


dùm) bap- a) 3 iy 
nt areh r ha, üp” (W) = 2H) - SU, 1w). (8.2) 


This yields a sequence in time of parameterized linear systems of the form 


Eg _ on 


N(u™* (W; p) ras =$ (p), n=0,... Ni -4 (8.3) 


where u”(u) € RN: and p"(H) < IR: denote the FOM vector representation of the 
velocity and the pressure, at time t,, u(y) = Uy € IR: is the initial condition, and 
N(u™* (p); p) € RN and g"*!(u) € IR* are given by 


a u n,* J T 
Niu" p) - [a A i (Hs) i 


1 u n,o +1 
aaea [a wou +f a! 
f3" (y) 


> 


(8.4) 


Here M” (u) € RAXM is the velocity mass matrix; D(p) € RMN and B(u) € RMN 
encode the velocity stiffness and the divergence operator, whereas C(u™* (u); u) € 
RMN arises from the linearization (about u”* (p) = 2u"(w)—u" 1(y)) of the nonlinear 
term. 

To deal with complex domains and their deformations in a flexible way, we exploit 
a general mesh deformation technique (see, e. g., [60]), in which domain deformations 
result from an additional FE problem providing either an harmonic [4] or an elastic 
[64, 62, 61] deformation by properly extending boundary displacements; see also [42] 
for further details. The corresponding meshes are also obtained as a deformation ofa 
reference mesh, hence not affecting the topology of the degrees of freedom. See also 
[13] for the case of a parameterized ROM for Navier-Stokes equations in domains with 
variable shapes, relying on an FOM based on a finite volume discretization, built for 
accelerating the calculation of pressure drop along blood vessels. 


8.2.2 Cardiac electrophysiology 


The propagation of the electric signal through cardiac cells is the main mechanism re- 
sponsible for their contraction, finally resulting in atrial and ventricular contractions. 
Mathematical models of cardiac electrophysiology describe the action potential mech- 
anism of depolarization and polarization of the cardiac cells, which consists of rapid 
variations of the cell membrane electric potential with respect to a resting potential. In 
particular, cellular models characterize the electric potential of a single cell, whereas 
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physiological models provide a quantitative description of the action potential prop- 
agation at the tissue level. The former are based on systems of ODEs that describe the 
variation of ionic species and ionic currents; the latter are derived from the former by 
means of suitable homogenization procedures. 

Here we are interested to characterize the evolution of the electric potential for 
a wide range of physical parameters affecting both electric conductivities at the tis- 
sue level and ion dynamics at the cell level. Coupling the monodomain model for the 
transmembrane potential u(w) with a phenomenological model for the ionic currents 
— here involving a single gating variable w(u) — in a domain Q c RÍ, d = 2,3, repre- 
senting, e. g., a portion of the myocardium (or the whole left ventricle) results in the 
following time-dependent nonlinear differential system: For each t € (0, T), 


ə 
u) - div(D(w)Vu()) + Bion (U(4), wH); H) = Iapp(ti) ino, 
i +gu(u) wu) p) =O ino, ea 
stl = on ðQ, 
on 


u(M) =Ug, w(p)=wọ inQ, att=0. 


Here t denotes a rescaled time, J,,,, is an applied current representing the initial acti- 


pp 
vation of the tissue, Ion and g model the cellular bioelectric activity, and d denotes the 
diffusivity tensor. For the case at hand, we aim at estimating the effect of (i) anisotropic 
conductivity, (ii) ion dynamics, and (iii) activation times on the electric conduction by 
parameterizing the tensor d, the functions Iion and g, and the source term Ipp» respec- 
tively. 

We model the cardiac tissue as composed of fibers (the cardiomyocytes) whose 
orientation varies from the epicardium to the endocardium due to the laminar organi- 
zation in sheets of the tissue [58]. At the macroscopic level, this structure yields pref- 
erential directions for the action potential traveling front. Therefore, at any point x, an 
orthonormal local reference system is described by the principal axes Ío (X), S9(X), and 
fig (X), with h (x) parallel to the fiber direction and with Sọ (%) and ño (X) orthogonal and 
tangent to the sheet direction. Denoting by oj, 0+, and g, the conductivity coefficients 
measured along the corresponding directions fo; So, and fig, the anisotropic conduc- 
tivity tensor is 


d(p) = 01 fo 8 fo + Ot So 8 So + Oy Tig 8 No. 


We assume that the left ventricle tissue is an axisymmetric anisotropic medium (0, = 
0,), so that the previous relation simplifies as follows: 


åm) =0,1+(o,- o) ho 8 fo» 


where the fibers’ structure is considered as (spatially dependent, but) y-independent. 
The reaction term Jon and the function g depend on both u and w, thus making the 


8 Reduced-order modeling for applications to the cardiovascular system =—— 257 


PDE and the ODE two-ways coupled. In this chapter, we focus on the Aliev-Panfilov 
model, for which 


Lon(u, w; H) = Ku(u — a)(u — 1) + wu, 


8.6 
stu wip) = (60+ EN w — Ku(u - b - 1)); aa 


the (parametric) coefficients K, a, b, £9, c4, and c, are related to the cell. 

Finally, we model the electric activation as the combination of three applied cur- 
rent stimuli, {ġ; BÈ ,, at three fixed spatial locations Rio with t,, t expressing a 
time delay 


Tapp (ts H) = $1(X) 10.2) (t) + PPh, +2] (t) + 3X) pe,.¢,42) (0) 


where I, (t) denotes the indicator function of the set A, that is, I,(t) = 1ift € A, and 
I,(t) = 0 otherwise. 

To summarize, we consider p = 9 parameters for this test case: the two conduc- 
tivities o, and o;; five parameters affecting the ionic model, K, a, b, c,, €; and two 
characteristic times t}, t, affecting the electric activation. 

Problem (8.5) is first discretized in space by means of the FE method, using linear 
FEs for the transmembrane potential; a semi-implicit, first-order, one-step scheme is 
then used for time discretization [19], in which the nonlinear vector Ion € R™ at time 
tn41 is calculated using the solution computed at the previous time t,,. This decouples 
the PDE from the ODE leading to a linear system to be solved at each time step. At each 
time step ¢,,n = 0,...,N,;—1, a system of N, (independent) nonlinear equations must 
be solved, arising from the backward (implicit) Euler method, under the form 


w(u) -At g(u"(w), w""'(Q);) =w), n= 0,...,N,-1, (8.7) 


given w? (py) = Wo(H). The so-called ionic current interpolation strategy is used to eval- 
uate the ionic current term, so that only the nodal values are used to build a (piecewise 
linear) interpolant of the ionic current. This yields a sequence in time of parameterized 
linear systems of the form 


Car 5 Ago Jugo t Lon (u(y), w! u) u) 


MW uu) +E), n=0,...,N;-1, 


(8.8) 


where u”(u) € R™ and w"(u) € R™ denote the FOM vector representation of the 
transmembrane potential and the gating variable, respectively, at time t„, and u = 
uo(H) € R`, w? = Wọo(H) € R™ are the initial conditions. Here M(p) € Rn is the 
mass matrix and A(p) € RM encodes the diffusion operator, whereas Hal c€ RM 
encodes the applied current at time ¢,,,;. 
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The major computational costs are entailed by assembling the terms I,,,, and g at 
each time step and by the solution of the linear system (8.8); strong constraints on 
the spatial mesh size have to be taken into account due to the propagation of very 
steep fronts [18, 49, 28], yielding a very large dimension N,. On its turn, the time step 
At is required to be sufficiently small to capture the fast dynamics characterizing the 
propagation of the electric signal [26]. 


8.3 Reduced-order modeling 


ROMs aim at reducing the computational complexity and costs entailed by the re- 
peated solution of PDE problems [10, 52]. In the case of parameterized PDEs, the RB 
method is a remarkable instance of ROM that allows to dramatically reduce the dimen- 
sion of the discrete problems arising from numerical approximation — from millions 
to hundreds, or thousands at most, degrees of freedom. Proper orthogonal decom- 
position (POD) is a general-purpose technique widely used to build RB spaces; later, 
a (Petrov—)Galerkin projection onto the RB space is employed to generate the ROM. 
Whenever cheaply computable a posteriori error bounds are available, greedy algo- 
rithms can be used as an alternative strategy. 

Parameterized blood flows in idealized cardiovascular geometries have been ad- 
dressed by means of ROMs in [37, 43] and in [5] by taking into account more complex 
(and computationally challenging) subject-specific configurations; in all these cases, 
solutions of Navier-Stokes equations are computed with respect to inflow and/or ge- 
ometrical parameters, however without exploiting efficient, purely algebraic hyperre- 
duction techniques. Applications to PDE-constrained optimization problems arising 
in the context of optimal design of prosthetic devices can be found, e. g., in [44, 36]. An 
application of POD to the analysis of transient turbulence in a stenosed carotid artery, 
however without dealing with parameterized flows and the solution of the problem 
for new parameter instances, has been considered in [29]. Except for few cases [12, 14, 
15, 21, 27], which however do not systematically explore parameter-dependent prob- 
lems, the application of the RB method to the simulation of the cardiac function in 
subject-specific configurations is work in progress. 

In this chapter we provide a quick overview of the way the most relevant difficul- 
ties related to the application of the RB method to the two problems introduced in the 
previous sections have been tackled. In both cases we will rely on POD for construct- 
ing the RB spaces and on suitable hyperreduction techniques to deal with the efficient 
assembling of nonlinear or nonaffinely parameter-dependent terms appearing in the 
ROM. 
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8.3.1 Navier-Stokes equations 


The most relevant issues related to the reduction of the parameterized Navier-Stokes 
system (8.1) for the application at hand are (i) the construction of flexible, low- 
dimensional shape parameterization to describe complex shapes in terms of few 
parameters and (ii) the stability of the ROM (in the sense of a suitable inf-sup stability 
condition) when dealing with the approximation of both velocity and pressure. 

Expressing the RB approximation of velocity and pressure fields at time t, as a 
linear combination of the RB basis functions, 


u"(u) = V,uyQo, pP“) = Vppy HD), (8.9) 


where V,, € RYN and Vp € RM” collect the (degrees of freedom of the) basis 
functions, a Galerkin projection yields the following Galerkin-RB problem: Given u € 
P, uy’, uy, find (uy? (u), pR (W) € R™ x IRN» such that u$ (u) = uyo and 


n,* ul ) nk 
Ny(V,Uy (9); p) Ened = §y tu), n=1,...,N,-1, (8.10) 
where uyo = V,,Uo, 
Qi pu ne f E 
aa san = | Be MH * Pa) A O] 
By(H) 
1 yu n,o n+1 (8.11) 
gy (H) = De (H) + fri g 
fyo (H) 


and 
Dy) = V, DWV, MpD =V,M“(WV,,  By() = VZ BQOV, 
Cy(V,uy* (u); u) = VIC uR" (W); WV, 
E D = VIE O, fyo) = VE qo. 


To construct the RB matrices V,, and V, we rely on POD - separately on velocity 
and pressure variables — by collecting snapshots of the FOM solution for a sample of 
selected parameter values 4; i = 1,...,n,, and computing, for n = 0,...,N; — 1, the 
solution of the FOM (8.3). In particular, POD is first performed with respect to the time 
trajectory (for a fixed u;) and then with respect to u. Three issues must then be taken 
into account: 

1. ROM stability. Performing, as in (8.10), a Galerkin projection onto the RB space 
built through the POD procedure above does not automatically ensure the stabil- 
ity of the resulting RB problem (in the sense of the fulfillment of an inf-sup condi- 
tion at the reduced level). This yields a potentially singular matrix N(u™* (u); y). 
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Several strategies can be employed to overcome this shortcoming; here we aug- 
ment the velocity space by means of a set of enriching basis functions computed 
through the pressure supremizing operator. For eachi = 1,...,n, and for each 
n = 1,..., N; we compute the so-called supremizers by solving 


X,, (Mt u) = BT (4) P" (H), (8.12) 


where X, € RiP’: encodes the scalar product over the velocity space. Then, 
POD is performed to extract an enriching basis Y, € Mis, which is then merged 
(through a Gram-Schmidt orthonormalization) with the columns of V,,. This strat- 
egy leads to an RB problem which is inf-sup stable in practice, but whose well- 
posedness is not rigorously proven [56, 7]. Another option to enforce the inf-sup 
stability would rely on a coarse algebraic least-squares RB method; however, this 
strategy has only been investigated in the case of steady Stokes problems so far; 
see [23]. 

2. Efficient assembling of nonlinear terms. Because of the u dependence induced by 
the geometry deformation, all the matrices and vectors in the ROM (8.10) depend 
nonaffinely on the parameter u; moreover, a critical issue is represented by the 
linearized term Cy(V,uy (u); p) appearing in (8.4). To assemble it efficiently, we 
rely on the matrix version of the discrete empirical interpolation method (DEIM). 
Such a procedure requires the evaluation of a sample of system (vectors and ma- 
trices) snapshots, followed by a POD on vectors and vectorized matrices and a 
further selection of a set of well-chosen interpolation points; see, e. g., [46]. The 
matrix DEIM (MDEIM) is another technique also employed to compute an approx- 
imated affine decomposition of the diffusion D(j), the pressure-divergence B(u), 
and the velocity mass M” (u) matrices. 

3. Low-dimensional parameterization of the computational domain. To deal with 
complex domains and their deformations in an extremely flexible way, we exploit 
a general mesh deformation technique, in which deformations result from an ad- 
ditional FE problem describing either the behavior of the structure with respect to 
given inputs or an harmonic extension of boundary data. These are called mesh- 
based variational methods, and are often referred to as solid-extension mesh mov- 
ing techniques. The corresponding meshes are also taken as a deformation of a ref- 
erence mesh, without affecting the topology of the degrees of freedom. Denoting 
by QP the computational mesh over which the state problem is solved, a deformed 
volumetric mesh is obtained as 0),(u) = {p € R? : X,(W) = Èn + AH), Xp, € o9}, 
where the nodes position is modified (so that Q,(u) conforms to the updated 
boundary) while keeping the mesh connectivity fixed; the domain deformation 
ap) can then be approximated by an RB method, before constructing the ROM 
for the fluid flow problem; see, e. g., [25, 42] for further details. 
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8.3.2 Monodomain system 


To ensure the efficiency of ROMs when dealing with the parameterized coupled 
monodomain-ionic model (8.5) a generalization of the POD approach is envisaged, 
requiring the construction of local RB spaces. Additionally, hyperreduction tech- 
niques such as DEIM and its matrix version MDEIM are employed to enhance the 
construction of nonaffine and nonlinear terms. Regarding the PDE system (8.8) for 
the transmembrane potential, we assume that the RB approximation of the trans- 
membrane potential at time ¢,, is expressed by a linear combination of the RB basis 
functions, 


u” (p) = Vuh (p) (8.13) 


where V € R" collects the (degrees of freedom of the) basis functions. 
If the gating variable has already been updated to its current value w”*! (p) at time 


t"*! by solving (8.7), the Galerkin-RB problem reads 


M 
(AH + aw Jug + Vion Wah, w p) 
ae (8.14) 
= Na), 2 =0,...Np-1 


where Ay(y) = V7 A(w)V and My(u) = V'M(u)V; also in this case, since the u de- 
pendence shown by these matrices is nonaffine, we rely on the MDEIM to get an ap- 
proximate affine expansion. 
Two issues arise when dealing with this problem: 
1. PDE-ODE coupling. We can take advantage of the DEIM to avoid the evaluation 
of the full-order array Iion € IR“, which would compromise the overall ROM effi- 
ciency. Hence, we approximate 


Tion(Vuy, Wh 3) ai Tion) z 2 6,(t"s u) Zq (8.15) 


once m « N, p-independent vectors and Z4 € R”, 1 < q < m, basis vectors 
have been calculated from a set of snapshots {lion (Vut (u*), w3! u“); u“), k = 
1L... Ns £ = 0,...,N; — 1}; p-dependent weights 0,:P > R are then computed 
by imposing m interpolation constraints. Basis vectors are computed by means of 
POD [16], whereas the set of points (in the physical domain) where interpolation 
constraints are being imposed are iteratively selected by employing the so-called 
magic points algorithm [9, 40]. 

The ionic term in the potential equation can then be approximated by 


V Ton Vu"(u), wW; u) = VOP O)! Lon (P Vu" (u), Pw w; p), 


Npxm mx1 
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where ® = [z,|...|Zn] € RY" and P = [ez |... lez] € RY, with 
ez, = [0,...,0,1,0,...,0]Ë € RY, T being the set of m interpolation indices 
T c {1,..., Np} with |Z| = m. Note that the matrix Op, = V’O(PT®)! is 
p-independent and can be assembled once for all. As a matter of fact, this pro- 
cedure also enables a reduction of the computational complexity entailed by the 
solution of the ODE system (8.7). Indeed, its pointwise approximation can be 
advanced in time only on the m degrees of freedom 74, ..., Zm, thus resulting ina 
reduced ODE system for the vector w”, = P’w" e€ R”. 

Finally, the ROM for the monodomain system (8.5) reads as follows: Given H € P, 
find (uk (u), wt (u) € R™ x R” such that ug (0) 0) = Vuo (u), we, = P’ wo(p), 
and, forn = 0,...,N;,- 1, 


Wh (D) - Atg(P Van G), wa (D; p) = Wh), 


( Mu + Ango Jup oD (8.16) 
T; n n+1/,.), = M(H) n Tyn+l 
+ Dplion(P Vuy, Win (p); H) = At Uy(H) +V Tapp): 


2. RB space construction. Relying on POD on global RB spaces yields accurate ap- 

proximations only if very large dimensions (up to some hundreds) N and m of 
the POD expansion and of the DEIM approximation, respectively, are considered, 
ultimately yielding negligible speedups compared to the FOM. Indeed, parameter- 
ized problems in cardiac electrophysiology might easily yield solutions showing 
a remarkable variability over the parameter space, because of the huge sensitivity 
of the solution with respect to variations of parameters representing conduction 
velocities, fiber structures, and tissue anisotropy. 
Multiple local subspaces must be generated when performing the RB approxima- 
tion of the PDE solution, and the DEIM approximation can be used for the nonlin- 
ear term. Approximating the whole solution set by a series of subspaces of smaller 
dimension results in a more efficient approach than building a single subspace 
of larger dimension. For this reason, clustering (or partitioning) algorithms are 
employed, prior to performing POD, aiming at collecting snapshots (of both the 
solution and the nonlinear terms) into clusters; then, a local RB is built for each 
cluster through POD. Here we consider the approach proposed in [2] based on the 
k-means algorithm, to address the construction of local ROMs in the state space 
and further extended in [65, 3, 1]; see, e. g., [47] for further details. 


Finally, we highlight that if other models of cellular bioelectric activity were consid- 
ered, based on either a single ODE (such as FitzHugh-Nagumo, Rogers—McCulloch, or 
Mitchell-Schaeffer models ) or a system of ODEs (such as the Fenton—Karma model), 
the construction of ROMs would not change; see, e. g., [17, 20] for a review. 
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8.4 Computation of wall shear stress in a carotid 
artery bifurcation 


The first application we deal with is related to blood flow in a subject-specific three- 
dimensional carotid bifurcation, in terms of both flow variables and derived outputs 
of interest. The carotid bifurcation is located along the sides of the neck and supplies 
blood to the face and the brain [66]; it is composed by the common carotid artery 
(CCA), which then splits in the internal carotid artery (ICA) and the external carotid 
artery (ECA) (Figure 8.1a). In adult age, the carotid bifurcation might be affected by 
atherosclerosis, that is, a progressive narrowing of the artery, which might ultimately 
lead to stroke. Blood dynamics play an important role in the development of such dis- 
ease, and one of the main indicators employed in the risk analysis is the distribution of 
the WSS on the vessel wall boundary T, close to the bifurcation [59]. Numerical sim- 
ulations can provide such quantitative results able to support clinicians and recent 
results are reported, e. g., in [35, 30]. The WSS is defined as 


Ty = (eü) -t = 2p(e()n - (etä - à)ñ), 


where ñ and t are the (outer) normal and tangential unit vectors on T„, respectively, € 
is the strain tensor, and v is the dynamic viscosity of the fluid. 

Here we exploit the proposed ROM work flow to investigate the behavior of blood 
flow in different virtual scenarios, described in terms of parameterized domains and 
inlet boundary conditions. As a result, the WSS distribution will depend on parame- 
ters, too. 
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Figure 8.1: Computational domain with common carotid artery (CCA), internal carotid artery (ICA), 
and external carotid artery (ECA) (left) and reference inlet flow rate Qcca (t) [cm?st] with highlighted 
systole, mid deceleration, and diastole phases (right). 
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We denote by I;,, the CCA inlet boundary portion, located at the bottom of the 
bifurcation in Figure 8.1a; Tout is given by the two ECA and ICA outflow boundaries, 
located on its top; finally, ry = 0O\(Tin UT out). By following the setup employed in [25], 
at the CCA inlet boundary we prescribe a parameterized flow rate Qcca(t; 4), obtained 
as a suitable modification of the reference flow rate Q¢¢,q(t), acquired from echo-color 
Doppler and reported in Figure 8.1b for a single heartbeat. The resulting inlet velocity 
Sys(H) is a parabolic function in the normal direction to I;,,(u) and vanishing in the 
tangential ones, such that 


| Bys(tsM) -ñ Tin = Qccalt; H) = Ho QXca(C). (8.17) 
Tin 


For the case at hand p = (42) € P = [0.2, 0.4] x [0.75, 1.0] c IR’; the param- 
eter 44 tunes the narrowing of the ICA by loading a stress proportional to 4, in the 
region shown in Figure 8.2a. This simulates the effect of a stenosis obstructing the 
vessel; the larger u4, the more emphasized the deformation. See [25] for a detailed de- 
scription on the way this geometrical parameterization is built. On the other hand, 
Hy determines the magnitude of the inlet flow rate; see (8.17). The radius at the inlet 
boundary at the entrance of the CCA measures approximately 0.27 cm, leading to a 
peak of the inlet velocity profile of approximately 59 cm s™*, when p = 1, during the 
systolic phase. Two examples of deformation (with respect to the reference domain) 
are reported in Figure 8.2 for different instances of the parameter u, = (0.38, 0.975) 
and y; = (0.21, 0.7625). Finally, the blood kinematic viscosity is v = 0.035 cm’s*; asa 
result, Re = 450. Taylor—Hood (IP? — P!) FEs are employed for the spatial discretiza- 
tion, leading to Ný = 248,019 degrees of freedom for the velocity and NP = 11,911 for 
the pressure, respectively; as a result, N, = N} + NP = 259,930. A BDF2 scheme with 
At = 0.01 has been considered for the time discretization, taking T = 0.64 seconds as 
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Figure 8.2: Region where the stress is loaded to deform the mesh (left) and examples of deformation 
for two instances of the parameter } = (0.38, 0.975) (center) and u, = (0.21, 0.7625) (right). 
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Figure 8.3: Velocity vector for 4 = (0.38, 0.975) at different time steps t = 0.2, 0.35, 0.5. 


final time to simulate an entire heartbeat. Numerical simulations have been carried 
out by employing 32 cores. 

Regarding system approximation, the MDEIM and DEIM applied to the arrays ap- 
pearing in (8.11) yield 277 matrix operators and 16 right-hand side vectors. Regarding 
state reduction, the RB matrices V,,, Vp, Vs for velocity, pressure, and supremizing 
functions, respectively, are built with POD; this latter retains N, = 836, N, = 506, and 
N, = 742 basis functions, respectively. Compared to the dimension N, = 259,930 of the 
original FOM, the reduction in the system size is of about 125. 

Examples of ROM solutions for different values of the parameters, computed with 
the ROM, are reported in Figures 8.3 and 8.4, with an error of the order of 0.1 % if com- 
pared with the FE simulation. On average, the Navier-Stokes equations are solved by 
the ROM with a computational cost of 4.05 seconds per time step, yielding a speedup 


Velocity Magnitude 
10 


Figure 8.4: Velocity vector for u, = (0.21, 0.7625) at different time steps t = 0.2, 0.35, 0.5. 


1 Computations have been performed on the Piz-Daint cluster at the Swiss National Supercomputing 
Center with Intel® Xeon® E5-2695 v4 @ 2.10 GHz and 64 Gb RAM. 
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Figure 8.5: Computed flow rate at ICA (left) and ECA (right) for wz, = (0.38, 0.975), 
Hy = (0.35, 0.9375), u3 = (0.21, 0.7625), u, = (0.38, 0.9375), u; = (0.38, 0.7625). 


factor of about 20 with respect to the FOM. The significant gain in the speedup allows 
for the real-time simulation performed by clinicians, since in only few minutes the 
computation can be carried out for any new scenario. 

The velocity pattern is affected by the parameter values, also influencing the flow 
rate at the outlet boundaries: In Figure 8.5 we report the outflow rate at the ICA and 
ECA boundaries as a function of time, for different parameter values. The physical 
parameter py mainly impacts the absolute value of the flow rate, whereas the geomet- 
rical parameter u, mostly affects the way blood flows split into the two branches: the 
smaller y,, the larger the flow rate through the ICA. 

Finally, in Figures 8.6 and 8.7 the WSS magnitude distribution is reported for dif- 
ferent values of the parameters and times; as expected, the WSS is higher during the 
systolic peak and concentrated close to the bifurcation. 
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Figure 8.6: WSS for pt, = (0.38, 0.975) at different time steps t = 0.2, 0.35, 0.5. 


8 Reduced-order modeling for applications to the cardiovascular system —— 267 


WSS Magnitude WSS Mogntude , WSS Mogntude 
28.7 


20 

10 | 
0 

” 

Z x 


Figure 8.7: WSS for u, = (0.21, 0.7625) at different time steps t = 0.2, 0.35, 0.5. 


8.5 Evaluation of activation maps in cardiac 
electrophysiology 


The second application we consider is related to the efficient evaluation of activation 
maps over a subject-specific left ventricle geometry. These outputs are the virtual coun- 
terpart of epicardial or endocardial potential recordings (electrograms) obtained by 
means of (unipolar or bipolar) catheters located inside a cardiac chamber, or on its 
external surface. Common relevant features extracted from these measurements are, 
e. g., voltage maps, showing the distribution of the electric potential at any given time, 
and activation maps, providing information about the time when the electric wave- 
front reaches a given point. These information are crucial in the clinical practice, for 
instance when treating cardiac arrhythmias by radio-frequency catheter ablation. Tar- 
get sites for ablation, often consisting of slow-conducting narrow isthmuses bordered 
by areas of scar tissue, are indeed identified by endocardial voltage and activation 
maps. 

The efficient numerical evaluation of several different scenarios in terms of elec- 
tric potential, by means of accurate ROMs, can open new paths to address the propa- 
gation of input uncertainty on the output and the systematic evaluation of the impact 
of (optimized) standard intervention procedures, aiming at optimizing them. For the 
case at hand, we consider the monodomain model (8.5) with the Aliev-Panfilov model 
(8.6) for estimating the effect of (i) anisotropic conductivity, (ii) ion dynamics, and (iii) 
activation times on the electric conduction. 

We consider p = 9 parameters for this test case: the two conductivities 0; € [12.9 - 
0.1, 12.9 - 0.15] and g; € [12.9 - 0.05, 12.9 - 0.1]; five parameters affecting the ionic model, 
K e [7,9], a,b € [0.05, 0.15], c, € [0.1,0.2], and € € [0.005, 0.02]; and finally, two 
parameters affecting the electric activation, t, € [5,10] and t, € [10,15]. Regarding the 
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Figure 8.8: Left ventricle geometry: activation points (left), computational mesh (center), and fiber 
orientation (right). 


latter, the three applied current stimuli (Pi, are 


3 372 
IX — Xill 
4 


(X) = 1 ; exp( ) i= 1,2,3. 
2(27) 2 

The location of the three activation points, together with the computational mesh 
of the subject-specific left ventricle geometry we consider, and the fiber orientation 
are reported in Figure 8.8c. The left ventricle geometry, extracted from the atlas of 
cardiac geometries described in [31], has been discretized using a three-dimensional 
mesh with N, = 24,660 vertices and 105,904 elements. The time interval is [0,600 ms] 
and the time step is At = 0.25 ms. A single query to the FOM, based on linear FEs, takes 
about 23.5 minutes to be computed.” 

We assess the computational performance and the accuracy of the ROM built by 
considering the k-means algorithm to address the construction of local ROMs in the 
state space. We start from a training sample of 25 parameter vectors, leading to a set 
of training snapshots of dimension N, x 6- 10. The 15 clusters formed by the k-means 
algorithm subdivide the snapshots mainly with respect to time. Indeed, as shown in 
Figure 8.9, the clusters’ centroids (or barycenters) reflect, roughly speaking, the evo- 
lution of the electric potential over the time interval. 

We then compute the relative error 


N, 
a ju” 7 uilo 
ey = 


N, 2 
Taiao 


> 


where || - lao) denotes the H 1(Q)-norm, on three selected test parameter vectors, 


Hi = [1.29, 1.29, 7.023, 0.0837, 0.1162, 0.0179, 10, 10] r (8.18) 


2 All timings are obtained by performing calculations on an Intel(R) Core i7-8700K CPU with 64 Gb 
DDR4 2666 MHz RAM. 
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Figure 8.9: Centroids computed by the k-means algorithm on the training set. 


HS = (1.3968, 1.0333, 7.5259, 0.1154, 0.1689, 0.0175, 7.2527, 10.4191)", (8.19) 
46 = (1.4377, 1.2341, 7.3048, 0.1326, 0.1538, 0.02, 5.3909, 12.2134] z (8.20) 


which we also employ for the sake of visualization. Figure 8.10 shows the action po- 
tential for u = u; , i = 1,2,3, at different time steps t = 60,100,320 ms. 


Table 8.1: Computational performance on the three test parameter vectors H» i= 1,2,3. 


Parameters FOM time ROMtime Relative error Speedup 


Hi 1409s 26.15 0.0077 54x 
1} 1419s 33.25 0.005 43x 
3 1398s 29.45 0.0099 48x 


Table 8.1 shows that the local ROM provides an average speedup factor of about 48x 
compared to the FOM, ensuring at the same time a relative error smaller than 1%. 
The resulting POD bases on each cluster (obtained by considering a tolerance of 10°? 
on the relative energy content of the discarded POD modes) have dimension ranging 
from 38 to 162. The same procedure is applied within the DEIM for the approximation 
of the nonlinear terms; in this case the dimension of the POD bases (obtained with a 
tolerance of 10-8) ranges from 492 to 1221. 

We then show how to recover two outputs of clinical interest from the computed 
solutions: 
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t = 100 ms t = 320 ms 


Figure 8.10: Action potential for the three test parameter vectors u}, Í = 1,2, 3, at different time steps 
t = 60,100,320 ms. 


The former is the activation map, which is obtained by evaluating the local acti- 
vation time (LAT) at each vertex of the mesh; see Figure 8.11. The LAT at a spatial 
point is given by the time when the electric wavefront passes through that point, 
that is, when the maximum negative slope of the electric deflection is measured. 
Such a map allows one to understand if regions of slow conduction are present in 
the tissue, for instance. 

The latter is a set of six simulated electrocardiograms (ECGs), representing the 
unipolar precordial (or chest) leads V;,...,V¢; these signals can be numerically 
approximated by integrating the projection of the heart vector Vu onto the direc- 
tion vector V(1/|l7|l), i. e., 


Ve [ vu- v( = Jaa, EE 
Q 

Ži being the positions of the pseudo-electrodes [34, 57]. For the case at hand, we 
locate the pseudo-electrodes as reported in Figure 8.12 in order to mimic their po- 
sition on the chest. ECG is a noninvasive test which conveys a large amount of 
information about the heart conditions. Variations on the uncertain ionic coeffi- 
cients K, a, b, c4, and £ induce considerable changes in the T wave and the QT 
interval, as shown in Figure 8.13. 
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Figure 8.12: Position of the pseudo-electrodes used for the approximation of Vg, k = 1,...,6, and 
ECG scheme. 


Also in this case, outputs of interest can be inexpensively evaluated taking advantage 
of the significant gain in the speedup provided by the ROM. 


8.6 Conclusions and outlook 


In this chapter we have shown two examples of applications of state-of-the-art ROM 
techniques to relevant problems in cardiovascular modeling, dealing with (i) the ar- 
terial fluid dynamics and (ii) the electric activity of the heart. In both cases, we suit- 
ably combined projection-based ROMs, built through POD, and hyperreduction tech- 
niques to enhance the assembling of the reduced-order problem. This allows compu- 
tational speedup factors of about 20-50 with respect to the high-fidelity, FOM built 
by the Galerkin FE method, without substantially lowering the accuracy of the FOM 
approximation. The availability of efficient and reliable ROMs is thus of paramount 
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Figure 8.13: Numerical approximation of unipolar precordial leads V;, k = 1,...,6, for u} , i = 1,2,3. 


importance to enable parametric studies, sensitivity analysis, and uncertainty quan- 
tification in complex, subject-specific scenarios (e. g., [45, 32, 50, 55]), yet unaffordable 
with standard, high-fidelity techniques even on modern parallel architectures. 


Despite huge efforts to enhance ROM efficiency still preserving their accuracy, sev- 


eral challenges remain open when dealing with cardiovascular applications. A nonex- 
haustive list includes, among others: 


to achieve fine temporal and spatial resolution, the training phase of ROMs (of- 
fline) as well as hyperreduction techniques becomes computationally intensive; 
new approaches are needed to address multiphysics and multiscale models, in- 
volving a wide range of spatio-temporal scales, more particularly to handle cou- 
pled problems; 

a full decoupling between the ROM and the FOM can be rather involved to obtain 
in the case of fine computational meshes and complex geometries, when dealing 
with hyperreduction techniques; 

intra-patient and inter-patient variability involves complex parameterizations of 
the model inputs; 

nonlinearity and high sensitivity of the solution with respect to parameter varia- 
tions limit computational speedups. 


Very often, several of these criticalities are simultaneously present in cardiovascular 
models, thus making the design of efficient and accurate ROMs a critical task. 
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On the other hand, an emerging strategy in the field of cardiovascular applications 
is grounded on the use of purely data-driven surrogate models or emulators, based, 
e. g., on machine learning (ML) techniques (e. g., [54, 48, 24, 22, 33]), like artificial neu- 
ral networks, or Gaussian process regression. This approach is especially relevant to 
approximate input-output maps featuring low-dimensional outputs as quantities of 
interest. Despite their efficiency at testing time (once a training phase, usually expen- 
sive, has been performed) for the sake of output evaluations for new parameter in- 
stances, often these techniques lack interpretability. Besides, the lack of error indica- 
tors makes their construction tailored on the specific problem at hand. In this respect, 
the combination of physics-based models (among which we also include projection- 
based ROMs) with data-driven, ML-based techniques for the sake of efficiency looks 
promising in view of a future translation of ROMs in clinical practice. 
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Jean-Christophe Loiseau, Steven L. Brunton, and Bernd R. Noack 
9 From the POD-Galerkin method to sparse 
manifold models 


Abstract: Reduced-order models are essential for the accurate and efficient prediction, 
estimation, and control of complex systems. This is especially true in fluid dynamics, 
where the fully resolved state space may easily contain millions or billions of degrees 
of freedom. Because these systems typically evolve on a low-dimensional attractor, 
model reduction is defined by two essential steps: (1) identifying a good state space 
for the attractor and (2) identifying the dynamics on this attractor. The leading method 
for model reduction in fluids is Galerkin projection of the Navier-Stokes equations 
onto a linear subspace of modes obtained via proper orthogonal decomposition (POD). 
However, there are serious challenges in this approach, including truncation errors, 
stability issues, difficulty handling transients, and mode deformation with changing 
boundaries and operating conditions. Many of these challenges result from the choice 
of a linear POD subspace in which to represent the dynamics. In this chapter, we de- 
scribe an alternative approach, feature-based manifold modeling (FeMM), in which 
the low-dimensional attractor and nonlinear dynamics are characterized from typical 
experimental data: time-resolved sensor data and optional nontime-resolved parti- 
cle image velocimetry (PIV) snapshots. FeMM consists of three steps: First, the sensor 
signals are lifted to a dynamic feature space. Second, we identify a sparse human- 
interpretable nonlinear dynamical system for the feature state based on the sparse 
identification of nonlinear dynamics (SINDy). Third, if PIV snapshots are available, 
a local linear mapping from the feature state to the velocity field is performed to re- 
construct the full state of the system. We demonstrate this approach, and compare 
with POD-Galerkin modeling, on the incompressible two-dimensional flow around a 
circular cylinder. Best practices and perspectives for future research are also included, 
along with open-source code for this example. 
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9.1 Introduction 


Understanding, modeling, and controlling complex fluid flows is a central focus in 
many scientific, technological, and industrial applications, including energy (e. g., 
wind, tidal, and combustion), transportation (e. g., planes, trains, and automobiles), 
security (e. g., airborne contamination), and medicine (e. g., artificial hearts and ar- 
tificial respiration). Improved models of engineering flows have the potential to dra- 
matically improve performance in these systems through optimization and control, 
resulting in practical gains such as drag reduction, lift increase, and mixing enhance- 
ment [38, 21, 98, 85, 25]. Although the Navier-Stokes equations provide a detailed 
mathematical model, this representation may be difficult to use for engineering de- 
sign, optimization, and control. Instead, they are commonly discretized into a high- 
dimensional, nonlinear dynamical system with many degrees of freedom and mul- 
tiscale interactions. These equations are nonetheless expensive to simulate, making 
them unwieldy for iterative optimization or in-time control. They may also obscure the 
underlying physics, which often evolves on a low-dimensional attractor [49, 77]. The 
various fidelities of model description were described by [115]: white-box describes an 
accurate evolution equation based on first principles (e. g., Navier-Stokes discretiza- 
tion), gray-box describes a low-dimensional model approximating the full state (e. g., 
proper orthogonal decomposition [POD]-Galerkin models), and black-box describes 
input-output models that lack a connection to the full state space (e. g., neural net- 
works). 

In the following, we outline related reduced-order models as our point of depar- 
ture in Section 9.1.1 and foreshadow proposed innovations of this study in Section 9.1.2. 


9.1.1 Related reduced-order models as point of departure 


Reduced-order models provide low-dimensional descriptions of the underlying fluid 
behavior in a compact and computationally efficient representation. This is illustrated 
in Figure 9.1, where, starting from full-state velocity snapshots obtained from direct 
numerical simulation, one extracts the leading coherent structures in order to obtaina 
low-dimensional representation of the system’s dynamics. There are many techniques 
for reduced-order modeling, ranging from physical reductions to purely data-driven 
methods, and nearly everything in between. POD [100, 14, 49] provides a low-rank 
modal decomposition of fluid flow field data, extracting the most energetic modes. 
It is then possible to Galerkin project the Navier-Stokes equations onto these modes, 
resulting in an approximate, low-dimensional model in terms of mode coefficients [78, 
28]. POD-Galerkin models are widely used, as they are interpretable, gray-box models, 
and it is straightforward to reconstruct the high-dimensional flow field from the low- 
dimensional model via POD modes. The first pioneering example of [4] featured wall 


9 Sparse feature-based manifold modeling —— 281 


Navier-Stokes simulation Dimensionality reduction Simple representation 


Figure 9.1: Illustration of reduced-order modeling. Starting from a direct numerical simulation of 
the Navier-Stokes equations (left), the dominant spatio-temporal coherent structures are extracted 
from a set of velocity snapshots (center). The temporal evolution of these structures then provides a 
simplified representation of the system’s dynamics (right) amenable to modeling. 


turbulence, almost three decades ago. Subsequent POD models have been developed 
for the transitional boundary layer [83], the mixing layer [111, 114], the cylinder wake 
[33, 42], and the Ahmed body wake [80], to name only a few. 

POD-Galerkin modeling is challenging for changing domains [18], changing 
boundary conditions [45], and slow deformation of the modal basis [5]. Standard 
Galerkin projection can also be expected to suffer from stability issues [82, 90, 29], 
although including energy-preserving constraints may improve the long-time stability 
and performance of nonlinear models [7, 31]. POD-Galerkin models tend to be valid 
for a narrow range of operating conditions, near those of the data set used to gen- 
erate the POD modes. Transients also pose a challenge to POD modeling. Refs. [77] 
and [106] demonstrate the ability of a low-dimensional model to reproduce nonlinear 
transients of the von Karman vortex shedding past a two-dimensional cylinder, pro- 
vided the projection basis includes a shift mode quantifying the distortion between 
the linearly unstable base flow and marginally stable mean flow. These techniques 
have been extended to include the effect of wall actuation [45, 81]. 

In addition to the physics-informed Galerkin projection, data-driven modeling 
approaches are prevalent in fluid dynamics [21, 85]. For example, dynamic mode 
decomposition (DMD) [50, 86, 55], the eigensystem realization algorithm (ERA) [51], 
Koopman analysis [72, 73, 109, 116], cluster-based reduced-order models [53], NAR- 
MAX models [15, 95, 120, 44], and network analysis [76] have all been used to identify 
dynamical systems models from fluids data, without relying on prior knowledge of 
the underlying Navier-Stokes equations. DMD models are readily obtained directly 
from data, and they provide interpretability in terms of flow structures, but the re- 
sulting models are linear, and the connection to nonlinear systems is tenuous unless 
DMD is enriched with nonlinear functions of the data [116, 55]. Neural networks have 
long been used for flow modeling and control [74, 122, 56, 54], and recently deep 
neural networks have been used for Reynolds-averaged turbulence modeling [59]. 
However, many machine learning methods may be prone to overfitting, have limited 
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interpretability, and make it difficult to incorporate known physical constraints. Par- 
simony has thus become an overarching goal when using machine learning to model 
nonlinear dynamics. In the seminal work of [16] and [91] governing dynamics and 
conservation laws are discovered using genetic programming along with a Pareto 
analysis to balance model accuracy and complexity, preventing overfitting. 

Recently, [22] introduced the sparse identification of nonlinear dynamics (SINDy), 
which identifies parsimonious nonlinear models from data. SINDy follows the prin- 
ciple of Ockham’s razor, resting on the assumption that there are only a few impor- 
tant terms that govern the dynamics of a system, so that the equations are sparse 
in the space of possible functions. Sparse regression is then used to efficiently de- 
termine the fewest terms in the dynamics required to accurately represent the data, 
preventing overfitting. Because SINDy is based on linear algebra (i.e., the nonlin- 
ear dynamics are represented as a linear combination of candidate nonlinear func- 
tions), the method is readily extended to incorporate known physical constraints [61]. 
In general, it is possible to obtain nonlinear models using genetic programming or 
SINDy on POD or DMD mode coefficients, which make these methods gray box, hav- 
ing a transformation from the model back to the high-dimensional, interpretable state 
space. However, models developed on POD/DMD mode coefficients may still suffer 
from fundamental challenges of traditional POD-Galerkin models, such as captur- 
ing changing boundary conditions, moving geometry, and varying operating condi- 
tion. 


9.1.2 Contribution of this work 


In this work, we introduce a new gray-box modeling procedure that yields inter- 
pretable nonlinear models from measurement data. The method is applied to the 
well-investigated two-dimensional transient flow past a circular cylinder with slow 
change of the base flow and varying coherent structures [105]. In particular, we de- 
velop sparse interpretable nonlinear models only from the temporal amplitudes a,(t) 
and a,(t) of the leading vortex shedding POD modes, hereafter denoted as our fea- 
tures. Second, a sparse dynamical model is identified in this feature space. For the 
following step, full-state measurement data are assumed to be available. Combining 
the nonlinear correlations existing between the various POD modes with techniques 
from Grassmann manifold interpolation enables us to obtain highly accurate esti- 
mates of the flow field both in the vicinity of the linearly unstable base flow and the 
marginally stable flow. This mapping provides significantly more accurate flow recon- 
struction, as compared to a POD-Galerkin model of the same order. To summarize, 
the resulting gray-box modeling procedure has the following beneficial features: (i) it 
captures nonlinear physics, (ii) it is based on a simple, noninvasive computational 
algorithm, (iii) the resulting model is interpretable in terms of nonlinear interaction 
physics and generalized modes (optional with full-state data), and (iv) modeling 
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feature vectors is more robust to mode deformation, moving geometry, and varying 
operating condition. 

The chapter is organized as follows: Section 9.2 provides an overview of the flow 
configuration considered in this work, namely, the incompressible, two-dimensional 
flow past a circular cylinder at Re = 100. Based on velocity snapshots obtained from 
direct numerical simulations, two different reduced-order modeling strategies are pre- 
sented in Sections 9.3 and 9.4. First, Section 9.3 introduces the canonical POD-Galerkin 
reduced-order model and discusses its main limitations. Then, Section 9.4 presents a 
highly accurate low-order model identified using recent advances in machine learn- 
ing. Finally, Sections 9.5 and 9.6 summarize our key findings, highlight some connec- 
tions with previous works, and provide the reader with good practices and possible 
future directions to extend this work. 

This contribution closely relates to three chapters of Volumes 1 and 2 of this hand- 
book. Starting point is the POD-Galerkin method [12, Chapter 2]. A transient cylin- 
der wake illustrates the benefits from manifold interpolation [124, Chapter 7]: A two- 
dimensional manifold is more accurate than a POD expansion with 50 modes. The 
resulting dynamical system on this manifold is significantly simplified by SINDy [12, 
Chapter 7]. 


9.2 Benchmark configuration and dynamics 


The flow configuration considered is the canonical two-dimensional incompressible 
viscous flow past a circular cylinder at Re = 100, based on the free-stream velocity U,,, 
the cylinder diameter D, and the kinematic viscosity v. This Reynolds number is well 
above the critical Reynolds number (Re, = 48) for the onset of the two-dimensional 
vortex shedding [118, 104, 94] and below the critical Reynolds number (Re, = 188) for 
the onset of three-dimensional instabilities [119, 8, 117]. Its dynamics are governed by 
the incompressible Navier-Stokes equations 


ou 12 
— +V. =-Vp+ —V°u, 
Jt +V. (ugu) pt Be u (91) 


V-u=0, 


where u = (u,v)! and p are the velocity and pressure fields, respectively. The center of 
the cylinder has been chosen as the origin of the reference frame x = (x,y), where x 
denotes the streamwise coordinate and y denotes the spanwise coordinate. This study 
considers the same computational domain as in [77, 61, 63], extending from x = —5 to 
x = 15 in the streamwise direction and from y = -5 toy = 5 in the spanwise direction. 
A uniform velocity profile is prescribed at the inflow, a classical stress-free bound- 
ary condition is used at the outflow, and free-slip boundary conditions are used on 
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the lateral boundaries of the computational domain. The open-source spectral ele- 
ment solver NEK5000 [41] is used to solve the equations with a third-order accurate 
temporal integration. For the sake of reproducibility, all of the files required to rerun 
the simulations presented in this work are freely available at the following address: 
https://www.github.com/loiseaujc along with an illustrative Jupyter Notebook. 


9.2.1 Direct numerical simulation 


Figure 9.2 depicts the evolution of the lift coefficient C; as a function of time. This direct 
numerical simulation (DNS) has been initialized with 


u(x, 0) = u, + ER(U)(X), 


where u, is the linearly unstable base flow and R(u)(x) is the real part of the linearly 
unstable eigenmode normalized such that its amplitude is equal to unity (see Section 
9.2.2 for more details). The parameter e, fixing the initial amplitude of the perturba- 
tion, was set such that the initial energy of the perturbation is of the order 10°°. 


0.25 


0.00 


C(t) 


0 50 100 150 
t 


Figure 9.2: Time series of the instantaneous lift coefficient C, (t), from the linearly unstable base 
flow to the marginally stable mean flow, obtained by direct numerical simulation. The black dashed 
line depicts the exponential growth predicted by linear stability analysis while the gray shaded 
region highlights the window over which flow snapshots have been collected for the POD analysis 
presented in Section 9.3. 


Three different phases are clearly visible in the time evolution of C;(t), namely, a pe- 
riod of exponential growth for O < t < 60, the onset of nonlinear saturation for 
60 < t < 100, and finally the constant amplitude quasi-harmonic oscillatory regime 
for t > 100 characteristic of the von Karman vortex street. The nonlinear saturation 
mechanism is briefly described hereafter. The nonlinear interaction of the instabil- 
ity mode with itself produces Reynolds stresses that distort the underlying base flow 
which, in turn, modifies the shape of the instability mode. This distortion also induces 
a frequency shift, the flow oscillating at a frequency almost 30 % larger in its final sat- 
urated state compared to that predicted by linear stability analysis of the base flow. 
This process continues until an equilibrium is achieved, balancing the influence of 
the perturbation’s Reynolds stresses onto the instantaneous mean flow and the feed- 
back this mean flow has onto the instantaneous growth rate of the perturbation. When 
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this equilibrium is reached, the flow is in a marginally stable state [9] and the ampli- 
tude of the perturbation no longer grows. For a complete description of this stabilizing 
nonlinear feedback mechanism, interested readers are referred to the self-consistent 
model presented in [68] or the weakly nonlinear analyses conducted by [96] and [27]. 


9.2.2 Stability of the steady solution 


Given a fixed point u, of the Navier-Stokes equations, the dynamics ofan infinitesimal 
perturbation u’ evolving in its vicinity are governed by 


ou’ 1 1 1 1 o2 

— +V. (UĻ89U +U @Uu,)=-Vp +—VuU, 

T b) = VP + e (9.2) 
V-u' =0. 


Introducing the normal mode ansatz u' (x, t) = i(x)e“, this set of equations can be 
recast into the following generalized eigenvalue problem: 
u 


P 


The linear stability of the base flow u, is then governed by the real part of the eigen- 
value A. In the rest of this work, the linearly unstable flow u, has been obtained us- 
ing the selective damping approach [1] while the eigenpairs of the linearized Navier- 
Stokes operator have been computed using a time stepper Arnoldi algorithm [37, 6, 
60, 62]. Interested readers are referred to [30, 108, 97] for exhaustive reviews about 
hydrodynamic instabilities. 

The vorticity field of the linearly unstable base flow u, at Re = 100 is depicted 
in Figure 9.3a. To the best of our knowledge, this is the only fixed point of the Navier— 
Stokes equations known for this flow configuration. Its linear stability has been exten- 
sively investigated [43, 96, 68, 27], and it is now well known that the bifurcation oc- 
curring at Re, ~ 48 is a supercritical Andronov—Poincaré—Hopf bifurcation eventually 
giving rise to the canonical Bénard—von Karman vortex street. The vorticity field of the 
corresponding unstable eigenmode is shown in Figure 9.3b. This complex—conjugate 
pair of eigenmodes is the only unstable pair before the onset of three-dimensionality. 

From a dynamical system point of view, one thus concludes that, although our 
discretized system is of the order 10° dimensions, the unstable linear subspace of the 
fixed point is only two-dimensional, i. e., only two degrees of freedom are required to 
describe the evolution of the system within this linear subspace. Let us furthermore 
consider the following stable and unstable manifold theorem [46]. 


(9.3) 


0 O 


i 0) 
V- 0) 


7 eee ae -V 


Theorem 1. Let E be an open subset of R” containing the origin, let f € C\(E), and let p; 
be the flow of the nonlinear system 


da 
dt = f(a). 
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Suppose that f(0) = O and that the Jacobian matrix L = Df (0) has k eigenvalues with 
negative real part and n — k eigenvalues with positive real part. Then, there exists a 
k-dimensional manifold W° tangent to the stable subspace E° of the linear system 


at Ay = 0. Similarly, there exists an (n — k)-dimensional unstable manifold W“ tangent 
to the unstable subspace E". 


This theorem is of crucial importance for the understanding of the reduced-order 
model to be discussed in Section 9.4. Indeed, although we will eventually consider 
the nonlinear evolution of our 10°-dimensional system, we will see that this evolution 
can be described by a very simple dynamical system evolving onto a two-dimensional 
parabolic manifold originating from the aforementioned unstable subspace E". 


9.2.3 Stability of the mean flow 


For the flow configuration considered herein, the linearly unstable base flow u,(x) and 
the mean flow u(x) computed from DNS differ quite significantly from one another, no- 
tably in the size of the recirculation bubble (see Figures 9.3a and 9.4a). Consequently, 
predictions of the spatio-temporal characteristics of the fluctuation obtained by linear 
stability analysis of the base flow might be misleading. 


Figure 9.3: (a) Vorticity field of the linearly unstable base flow for the two-dimensional cylinder flow 
at Re = 100. (b) Real part of the leading unstable mode’s vorticity field. In both figures, blue shaded 
contours (solid lines) highlight regions of positive vorticity, while red shaded ones (dashed lines) 
highlight those of negative vorticity. In (a), a few streamlines are plotted (light gray) to highlight the 
extent of the recirculation bubble. (c) Eigenspectrum of the corresponding linearized Navier-Stokes 
Operator. 
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Figure 9.4: (a) Vorticity field of the marginally stable mean flow for the two-dimensional cylinder 
flow at Re = 100. (b) Real part of the marginal mode’s vorticity field. In both figures, blue shaded 
contours (solid lines) highlight regions of positive vorticity, while red shaded ones (dashed lines) 
highlight those of negative vorticity. In (a), a few streamlines are plotted (light gray) to highlight the 
extent of the recirculation bubble. (c) Eigenspectrum of the corresponding linearized Navier-Stokes 
operator. 


Even though the mean flow u(x) is not a solution of the stationary Navier-Stokes equa- 
tions, it has now become quite standard nonetheless to linearize the Navier-Stokes 
equations in its vicinity as to study its linear stability [9, 110, 11]. The eigenspectrum 
of the corresponding linearized Navier-Stokes operator is depicted in Figure 9.4c. As 
shown in [9], the leading eigenvalues have a zero real part, indicating that this mean 
flow is marginally stable. Moreover, while the frequency predicted by linear stability 
analysis of the base flow differs by almost 30 % from the one recorded in direct nu- 
merical simulation, the one predicted by stability analysis of the mean flow almost is 
a perfect match. This mismatch results from the strong distortion induced by the insta- 
bility mode as it saturates nonlinearly. Similarly, the eigenmode shown in Figure 9.4b 
provides a much better representation of the spatial characteristics of the fluctuations 
observed in DNS. For extensive details and theoretical justifications about mean flow 
stability analysis, interested readers are referred to [65, 9, 96, 68, 69, 110, 71, 11, 70] 
and references therein. 


9.3 POD-Galerkin projection of the Navier-Stokes 
equations 
POD [100, 49] provides a low-rank modal decomposition of fluid flow field data, ex- 


tracting the most energetic modes. It is then possible to project the Navier-Stokes onto 
the span of these POD modes, resulting in an approximate low-dimensional model 
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governing the evolution of the mode coefficients. POD-Galerkin models are widely 
used as they are interpretable gray-box models and it is straightforward to reconstruct 
the high-dimensional state vector of the original system from the low-dimensional 
model via the POD modes. The first pioneering example of [4] featured wall turbu- 
lence, over three decades ago. Subsequent POD models have been developed for the 
transitional boundary layer [83], the mixing layer [111, 114], the cylinder wake [33, 77, 
42], and the Ahmed body wake [80], to name only a few. In the present section, di- 
mensionality reduction via POD analysis is first presented in Section 9.3.1. Then, Sec- 
tions 9.3.2 to 9.3.5 discuss the derivation of the reduced-order model from the Navier-— 
Stokes equations and its properties, as well as its accuracy and limitations. 


9.3.1 Dimensionality reduction — POD analysis 


A large number of systems, including but not limited to fluid flows, are governed by 
high-dimensional nonlinear dynamics. Nonetheless, because most of these nonlinear 
dynamical systems are dissipative by nature, their dynamics are likely to evolve onto a 
lower-dimensional attractor characterized by a few dominant coherent structures con- 
taining a significant portion of the system’s energy [49]. Given a high-dimensional data 
set, the aim of dimensionality reduction is thus to extract a low-dimensional embed- 
ding capturing most of the variability of the original data. One of the most widely used 
techniques for dimensionality reduction is POD. It is also known as principal compo- 
nent analysis (PCA) in statistics and machine learning, as Kosambi-Karhunen—Loéve 
transform in signal processing, or as empirical orthogonal functions in meteorologi- 
cal science, and it is closely related to singular value decomposition (see Figure 9.5). 
For the sake of conciseness, the mathematical details of POD will not be discussed 
herein. For more details, interested readers are referred to [100] and [14]. Note addi- 
tionally that POD is discussed at length in this book series; see for instance Chapters 2 
and 12 of Volume 1. 

The gray shaded region in Figure 9.2 highlights the window over which snapshots 
of the base flow-subtracted fluctuation have been collected for the present POD anal- 
ysis at a sampling rate approximately 25 times higher than the circular frequency of 
the natural vortex shedding. Figure 9.6a depicts the fraction of the fluctuation’s kinetic 
energy captured by each of the first 10 POD modes along with its cumulative sum. Note 
that, because we have considered base flow-subtracted fluctuations rather than mean 
flow-subtracted ones, the leading POD mode corresponds to the shift mode [77]. This 
mode captures the distortion between the base flow and the mean flow (Figure 9.6b) 
and accounts for 46 % of the whole kinetic energy in our snapshots data set. Consid- 
ering the second and third POD modes, related to the vortex shedding (Figure 9.6c), 
97.7 % of the total kinetic energy is captured. Finally, less than 1% of the kinetic energy 
is discarded if one considers the first five POD modes, and less than 0.1% if the first 
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Figure 9.5: Schematic representation of the low-rank approximation of the data matrix Q by means 
of singular value decomposition. Each column of Q contains one snapshot obtained from direct nu- 
merical simulation. The matrix U contains the space-dependent POD modes u;(x) while V contains 
the associated temporal evolutions, with superscript H denoting the Hermitian (i. e., complex conju- 
gate transpose) operation. Finally, the diagonal matrix X contains the singular values whose square 
characterizes the amount of variance explained by the associated singular pairs. 
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Figure 9.6: (a) Fraction of the total variance (+) explained by each POD mode and the corresponding 
cumulative variance (x). This POD analysis has been performed using base flow-subtracted snap- 
shots collected during the gray shaded window in Figure 9.2. The zeroth POD eigenvalue in this plot 
is associated to the shift mode wy. Figures (b) to (e) depict the vorticity distribution of the shift mode 
and the first, third, and fifth POD modes, respectively. Only a subset of the whole computational 
domain is depicted. 
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seven ones are considered. For the sake of completeness, the vorticity field of selected 
POD modes are shown in Figure 9.6b-e. 

Figure 9.7 depicts the phase plots of these various POD modes. For Figure 9.7a—c, 
only the evolution of the flow once it has reached the limit cycle is shown. It can be 
seen that, within the (a4, a)-plane, the evolution of the flow traces a perfect circle 
underlining the periodic nature of the saturated vortex shedding for the Reynolds 
number considered. Additionally, the phase plots shown in Figure 9.7 highlight that 
the third and fourth POD modes correspond to the second harmonics of the vortex 
shedding, while the fifth and sixth modes capture its third harmonics. Finally, Fig- 
ure 9.7d shows the whole evolution of the system, from the base flow to the mean 
flow, projected onto the (a,, a,)-plane. As expected, one recovers the well-known low- 
dimensional parabolic manifold [77] characteristic of a large number of wake flows. 
It is these dynamics that we wish to capture in Section 9.3.2 using a POD-Galerkin 
reduced-order model. 


nA 
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ay 


Figure 9.7: Phase plots of various POD modes. For (a), (c), and (d), only the evolution once the flow 
has reached the limit cycle is depicted. In (b), the whole evolution is shown, from the linearly unsta- 
ble base flow to the marginally stable mean flow. 


9.3.2 Reduced-order modeling strategy — Galerkin projection 


The POD analysis performed in the previous section has revealed that close to 97.5 % 
of the base flow subtracted fluctuation’s kinetic energy is captured by considering 
only the shift mode and the first pair of POD modes. Starting from this observation, 
it thus appears reasonable to approximate the velocity field u(x, t) using the following 
Galerkin expansion: 


u(x, t) = u (X) + Uy(X)ag(t) + uy (X)a,(t) + u (x)a(t), (9.4) 


where u(x) is the linearly unstable fixed point of the Navier-Stokes equations, while 
u,(X), u(x) and u,(x) are the velocity fields associated with the shift mode and the first 
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two POD modes, respectively. Starting from the Navier-Stokes equations, our goal is 
thus to derive a low-dimensional system of nonlinearly coupled ordinary differential 
equations governing the evolution of the POD modes’ amplitudes a;(t). Introducing 
our Galerkin expansion ansatz into the Navier-Stokes equations and projecting the 
latter onto the span of our POD basis (this process is known as Galerkin projection), 
we obtain evolution equations for each amplitude a;(t) of the form 


da = X Lja; + > D3 Qi Aj Ay, (9.5) 
j j k 


with i, j, k = A, 1, 2. By convention, the coefficient a) associated to the base flow u, (x) 
is set to dy = 1. In the above equation, the linear term is given by 


Lj= (u 


while the quadratic one is 


1 2 
- V- (U, @ Uj + Uj Q Up) + Re” uw}, 


Qij = —(U;|V - (uj @ u,)), 
where (a|b) denotes the inner product 


(alb) = | a-bao. 
Q 


Note that, as in [77], we did not explicitly account for the pressure term. For the 
present case, this omission however hardly changes the prediction of the reduced- 
order model. For a detailed discussion about the importance (or insignificance) of the 
pressure term in POD-Galerkin projection reduced-order models, interested readers 
are referred to [79]. 


9.3.3 Does the model capture the key physics? 


Before discussing whether the reduced-order model derived by POD-Galerkin projec- 

tion is accurate or not, let us first investigate whether it captures the key physics of the 

problem. In the present case, this would imply that: 

1. The reduced-order model has a single fixed point located at a = 0. 

2. The unstable subspace E“ of the reduced-order model linearized in the vicinity of 
a = 0 is two-dimensional and associated with a complex-conjugate eigenpair. 

3. Ast — oo, the system eventually evolves toward a structurally stable limit cycle. 


It must be emphasized that if the reduced-order model fails to comply with any of these 
requirements, then it fails at capturing the key physics of the problem. 
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Given the low-dimensionality of the present model, condition 1 can easily be (and 
has been) checked by performing an extensive Newton search. As expected, the only 
fixed point admitted by our reduced-order model is a = O. The linearization of our 
model in the vicinity of this fixed point is given by 


da 

— = La, 

dee" 
s T 

witha = [a azan] and 


0.042 -0.986 0) 
L= | 0.959 0.046 0) 
0) O —0.047 


Spectral decomposition of this matrix reveals that its eigenvalues are 
A = {À = 0.044 + 10.972, A, = 0.044 — 10.972, A, = -0.047}, (9.6) 


while the corresponding set of eigenvectors is 


1 1 0 
Ey=44,=|-i|,a)=|i],a@,=|]o]}. (9.7) 
0 0 1 


Looking at these eigenpairs, it is clear that, as for the original Navier-Stokes equa- 
tions, the fixed point a = O of our POD-Galerkin reduced-order model is linearly un- 
stable. Moreover, its unstable subspace E“ is also two-dimensional and associated 
with complex—conjugate eigenvalues and eigenvectors corresponding to oscillatory 
dynamics in the (a4, a,)-plane while it is stable along the direction corresponding to 
the shift mode. Condition 2 is thus also fulfilled. 

The last condition that needs to be checked is whether or not the system natu- 
rally evolves toward a stable limit cycle as t — oo. To do so, we integrate in time our 
reduced-order model using a fourth-order accurate Runge-Kutta scheme. Figure 9.8 
depicts the predicted asymptotic evolution. As can be observed, this reduced-order 
model does evolve toward a stable limit cycle, although its amplitude is slightly 
larger than the amplitude of the limit cycle obtained from direct numerical sim- 
ulation of the Navier-Stokes equations. Our reduced-order model thus fulfills all 
three necessary conditions we stated at the beginning of this section and, as such, 
captures qualitatively the key physics of the two-dimensional cylinder flow. Conse- 
quently, the only question that remains to be answered is the following: How ac- 
curate is this reduced-order model? The answer to this question is the subject of 
Section 9.3.4. 
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Figure 9.8: Comparison of the limit cycles observed in DNS (—) and predicted by the three-POD mode 
reduced-order model (orange —-). 


9.3.4 How accurate is it? 


We have shown in the previous section that a reduced-order model derived from the 

Navier-Stokes equations by means of a POD-Galerkin projection procedure qualita- 

tively captures the key physics of the problem considered, namely: 

Property 1: It has a single fixed point at the origin. 

Property 2: This fixed point is linearly unstable and the associated unstable subspace 
is two-dimensional. 

Property 3: As t — oo, the reduced-order model predicts that the system naturally 
evolves toward a periodic limit cycle. 


Let us now try to further characterize the accuracy of said reduced-order model. In 

particular, we will focus our attention on two critical aspects: 

1. Does it appropriately capture the transient dynamics of the flow as it evolves from 
the linearly stable base flow to the marginally stable mean flow? 

2. How good are its flow reconstruction capabilities? 


As to answer to these questions, the reduced-order model is fed with a random initial 
condition having the same initial energy as that used in the direct numerical simula- 
tion described in Section 9.2, i.e., 


a(0) = aa, + Bap, 


such that laO) = 10°°. Figure 9.9 depicts the evolution of the fluctuation’s kinetic 
energy as a function of time obtained from direct numerical simulation and predicted 
by our POD-Galerkin reduced-order model. Although our low-order model qualita- 
tively captures the transient dynamics of the flow, i. e., a period of exponential growth 
followed by nonlinear saturation, it is clear that it largely overestimates the transients 
duration. Moreover, as nonlinear saturation occurs, the reduced-order model predicts 
an energy overshoot before it saturates at a level higher than that observed in DNS. 
These two observations put in the limelight two critical issues of a large number of 
reduced-order models derived from the Navier-Stokes equations by a POD-Galerkin 
procedure. 
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Figure 9.9: Evolution as a function of time of the kinetic energy €(t) of the base flow-subtracted 
fluctuation for the DNS and two Galerkin projection reduced-order models using either the first three 
or the first nine POD modes. 


Let us first consider the problem of the overestimation of the transients duration. This 
problem finds its roots in the major difference that exists between the POD modes as- 
sociated with the first harmonics of the vortex shedding and the eigenmodes of the 
linearized Navier-Stokes operator. Looking at Figure 9.3b and c, it can be seen that 
the POD modes are located further upstream compared to the instability modes. Con- 
sequently, while the projection of the linearized Navier-Stokes operator onto the span 
of the POD modes reasonably approximates the dynamics of the system in the vicinity 
of the mean flow, it provides a very crude approximation of the dynamics of the sys- 
tem when close to its fixed point, notably in terms of the instability growth rate. This is 
a structural problem of POD-Galerkin reduced-order models. Indeed, from a physical 
point of view, the instability modes continuously deform into the POD modes as the 
amplitude of the fluctuation grows. However, fixing the projection basis a priori using 
solely the POD modes prevents the reduced-order model from being able to capture 
this mode deformation and the continuous change of dynamics associated with it. As 
to alleviate this problem, [77] explicitly included the instability modes into the projec- 
tion basis. Although this trick partially solves the problem, it unnecessarily increases 
the dimensionality of the reduced-order model. 

The second problem of the present low-dimensional model is the energy over- 
shoot and the subsequent saturation to a higher level than the one observed in DNS. 
This problem arises from the projection of the Navier-Stokes equations onto a finite 
number of basis vectors and thus from the chosen truncation of the POD basis. In 
the present case, our projection basis consists only of the shift mode (quantifying the 
distortion between the base flow and the mean flow) and the POD modes associated 
with the first harmonics of the vortex shedding. Because of this choice, the energy 
cascade from the large scales to the small scales is truncated early on. As a conse- 
quence, the energy extracted by the leading POD modes from the underlying unstable 
base flow cannot be transferred correctly to smaller-scale structures, hence growing 
beyond their expected amplitudes and causing the energy overshoot observed in Fig- 
ure 9.9. This excess energy is eventually absorbed by the mean flow distortion until 
an equilibrium is reached, even though the final kinetic energy of the reduced-order 
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model nonetheless saturates at a higher level that the one observed in DNS. A naive 
approach to fix this issue would be to include more POD modes in the projection basis. 
This is illustrated in Figure 9.9, where the evolution of the kinetic energy predicted by 
a reduced-order model derived using a projection basis that includes the POD modes 
associated with the second, third, and fourth harmonics of the vortex shedding is also 
shown. Although increasing the rank of the POD basis from 3 to 9 mitigates the prob- 
lem, the energy overshoot still exists. Moreover, including these higher-order modes 
in the projection basis also modifies the properties of the linearized dynamics in the 
vicinity of the fixed point. In the present case, including the POD modes associated 
with the second harmonics of the vortex shedding actually increases the dimension- 
ality of the unstable subspace E“ from 2 to 4. In the vicinity of the fixed point, the prop- 
erties of the linearized reduced-order model thus become inconsistent with those of 
the linearized Navier-Stokes operator. 


9.3.5 Limitations of this approach 


Although the POD-Galerkin approach to reduced-order modeling has had consider- 

able success over the years, it nonetheless suffers from major limitations, even for a 

flow configuration as simple as the two-dimensional cylinder flow. For the case con- 

sidered herein, four major limitations can be listed: 

1. In order to accurately capture the dynamics of the system once on the limit cycle, 
the projection basis had to include a relatively large number of modes (i. e., eight) 
despite the simplicity of the dynamics, including very low energy modes. 

2. The low-dimensional system tends to exhibit an energy overshoot as nonlinear 
saturation occurs because of the truncation of the energy cascade. This trunca- 
tion of the energy cascade results from the projection of the nonlinear partial dif- 
ferential equations onto a finite set of basis vectors. 

3. Because of the difference between the linear instability and the POD modes ob- 
tained from the limit cycle, the reduced-order model largely overestimates the 
transients duration unless the instability modes are explicitly included into the 
projection basis. 

4. Finally, it can hardly account for the continuous mode deformation taking place 
as the flow evolves from the vicinity of the linearly unstable base flow to that of the 
marginally stable mean flow. A similar problem arises if one varies the Reynolds 
number slowly in time. 


Since the generalized mean field model of Noack et al. [77], various attempts have been 
made to limit these shortcomings. For instance, [99] and [113] used eddy viscosity mod- 
els to account for the added diffusion induced by the truncated modes, while [75] and 
[103] used linear interpolation to partially capture the continuous mode deformation. 
Recently, [34] have used sparse coding to obtain a nonorthonormal projection basis 
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for the turbulent lid-driven cavity flow that nonetheless included some of the small- 
scale structures needed for the energy cascade, while [40] combined POD-Galerkin 
projection with constrained convex optimization techniques to ensure that the sta- 
tistical properties of the POD amplitudes predicted by the reduced-order model were 
consistent with those obtained from direct numerical simulations. These works how- 
ever still had to include dozens of POD modes for numerical stability although the dy- 
namics of the system are lower-dimensional. Despite all these attempts to increase the 
range of validity of the POD-Galerkin projection approach, one must not forget that it 
still suffers from one critical limitation that cannot be overcome within this particular 
framework: The governing equations of the high-dimensional system (in our case the 
Navier-Stokes equations) need to be known before one even tries to perform model 
reduction. 


9.4 Manifold model 


The approach described in the previous section can be understood as a semi-empirical 
or partially data-driven approach. Indeed, while on the one hand the projection ba- 
sis is obtained via POD of a snapshots data matrix, the Galerkin projection procedure 
relies on a priori knowledge of the high-dimensional system’s governing equations. 
Let us now consider a fully data-driven model of the flow that leverages the existence 
of a low-dimensional nonlinear manifold. Starting from the POD analysis presented 
in the previous section, Section 9.4.1 illustrates how one can further reduce the di- 
mensionality of the problem by considering the nonlinear correlations existing be- 
tween the various POD mode amplitudes. As a second step, a low-dimensional sys- 
tem is obtained using recent system identification techniques in Section 9.4.2. Finally, 
given that the system under consideration evolves on a low-dimensional manifold, 
Section 9.4.5 highlights how one can use Grassmannian manifolds to solve the con- 
tinuous mode deformation problem when reconstructing the high-dimensional state 
vector of the full-order model, while Section 9.4.6 discusses some of the limitations of 
the approach proposed herein. 


9.4.1 Looking for nonlinear correlations 


PCA (equivalent to POD in mechanical engineering) is one of the most popular di- 
mensionality reduction techniques. One of the key reasons for this widespread us- 
age is that PCA finds its root in statistics. Moreover, when formulated as a singular 
value decomposition, PCA can be understood as an optimal low-rank matrix approx- 
imation and can thus leverage highly performing and scalable algorithms to handle 
extremely large data sets. Considering only the first few principal components (i. e., 
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the leading left singular vectors of the data matrix), one can define an optimal lin- 
ear subspace onto which the data can be orthogonally projected while minimizing 
(and quantifying) the amount of information lost in the process. From a statistical 
point of view, this orthogonal projection provides linearly uncorrelated features. De- 
spite its optimality properties, PCA unfortunately cannot unravel nonlinear correla- 
tions in the data and postanalyses are thus required. Accounting for such nonlinear 
correlations may however be beneficial to further reduce the dimensionality of the 
problem. 

Over the years, various alternatives have been proposed to overcome this major 
limitation in order to be able to capture nonlinear manifolds. One can cite for instance 
kernel PCA (kPCA) [92], Isomap [107], locally linear embedding (LLE) and its variants 
[84, 121, 35], spectral embedding [10], multidimensional scaling (MDS) [17], or all the 
variants of autoencoders recently reviewed in [13]. All these techniques are part of a 
domain now known as manifold learning or representation learning. However, for the 
particular problem considered herein, the dynamics are sufficiently simple so that we 
can assess the existence of nonlinear correlations directly from time series of POD 
modes’ amplitudes. From a practical point of view, the existence of a clear pattern in 
a phase plot (a;-a;) implies the existence of such nonlinear correlations (see Figure 9.7 
for examples). 

The POD analysis performed in Section 9.3.1 has revealed that less than 0.1% of 
the total kinetic energy in our training data set is discarded if we only consider the shift 
mode and the first six POD modes. Given the Fourier-like nature of the POD coefficients 
once the flow evolves on the limit cycle, these can be approximated by 


ay(t) = Ay, 
a,(t) + ia,(t) = å et" 
a;(t) + ia,(t) = A,e*, 


a(t) + iag(t) = Âge”, 


where w is the fundamental frequency of the vortex shedding, A, is the amplitude 
of the shift mode in the saturated stage, and Aj, A), and A; are the amplitudes of the 
first, second, and third pairs of POD modes, respectively. Guided by physical intuition, 
Figure 9.10 summarizes some of the possible triadic interactions arising from the non- 
linear convective term V - (u ® u) of the Navier-Stokes equations. Looking at these 
triadic interactions, it thus appears that the dynamics of the shift mode and of the 
second pair of POD modes both result from quadratic interactions of the first pair of 
POD modes with itself. Similarly, the dynamics of the third pair of POD modes result 
from the interaction of the first pair with the second pair of modes. Alternatively, this 
last quadratic interaction can also be understood as a cubic interaction of the first pair 
with itself. These intuitions are further confirmed by looking at the correlation matrix 
depicted in Figure 9.11. 
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Figure 9.10: Some of the possible triadic interactions arising from the nonlinear convective term V - 
(ugu) of the Navier-Stokes equations. These triadic interactions will guide us to determine the form 
of nonlinear correlations existing between the amplitudes of the various POD modes considered. 
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Figure 9.11: Pearson’s p correlation coefficient between various monomials of a, and a, and the am- 
plitude a, of the shift mode or the amplitudes a3 to ag of the higher-order POD modes. Blue denotes 
strong positive linear correlation, red denotes strong negative correlation, and white implies no 
linear correlation between the two variables considered. 


The exact form of these nonlinear correlations can be unraveled by polynomial regres- 
sion. Doing so, we obtain the following relationships: 


ay = 0.41(a7 + a): 


a; = —0.028(a; - a3) — 0.13a,a), 

a, = 0.065(a? — a3) — 0.056a, a), (9.8) 
ds = —0.065a;.a, + 0.0220, 

ag = -0.021a} + 0.066434. 


Figure 9.12 provides a comparison of the evolution of the various POD modes’ ampli- 
tudes obtained from DNS and the ones predicted by the nonlinear correlations iden- 
tified. As can be observed, these quadratic and cubic correlations accurately capture 
the evolution of the higher-order POD modes as well as the existence of the paraboloid 
manifold. Hence, it is clear that, although POD analysis reveals that seven POD modes 
need to be considered to accurately reconstruct the flow, only two of these modes are 
actual degrees of freedom of the system while the rest of them are entirely slaved to 
these two. This observation is consistent with the fact that, as shown in Section 9.2, 
the unstable subspace of the Navier-Stokes operator linearized in the vicinity of the 
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Figure 9.12: Same as Figure 9.7. The evolution of the coefficients a3 and a, predicted by the non- 
linear correlation models is also reported. In (d), only the parabola a, = 0.414? (i. e., a slice of the 
paraboloid manifold in the a, = 0 plane) is shown. 


unstable fixed point is only two-dimensional. The coming section is then devoted to 
the identification of the dynamical system governing the dynamics of a, and a). 


9.4.2 Low-dimensional system identification — SINDy 


Advanced regression methods from statistics, such as genetic programming or sparse 
regression, are driving new algorithms that identify parsimonious nonlinear dynam- 
ics from measurements of complex systems. Bongard and Lipson [16] and Schmidt 
and Lipson [91] introduced nonlinear system identification based on genetic program- 
ming, which has been used in numerous practical applications in aerospace engineer- 
ing, the petroleum industry, and finance. More recently, Brunton etal. [22] have pro- 
posed a system identification approach based on sparse regression known as sparse 
identification of nonlinear dynamics (SINDy). Following the principle of Ockham’s ra- 
zor, SINDy rests on the assumption that there are only a few important terms that gov- 
ern the dynamics of a given system so that the equations are sparse in the space of 
possible functions. Sparse regression is then used to determine the fewest terms in 
a dynamical system required to accurately represent the data. The resulting models 
are parsimonious, balancing model complexity with descriptive power while avoid- 
ing overfitting and remaining interpretable. For more details about SINDy, interested 
readers are referred to Chapter 12 of Volume 1 of the present book series as well as to 
the increasing body of literature on the subject [22, 66, 23, 87, 101, 32, 89, 88, 67, 61, 
63, 52, 24, 48]. 

The nonlinear correlation analysis conducted in the previous section has revealed 
that the only true degrees of freedom of the system are the POD amplitudes a, and a). 
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Thus, we now aim to find a nonlinear dynamical system 


da 

ar = fila, a), 

da (9.9) 
T = fila, a), 


where fı : R? > Rand f, : R? > Rare two unknown functions to be identified 
with SINDy. For the sake of simplicity, we will assume that these two functions are 
polynomial functions of a, and a. In general, any basis functions may be used in 
the SINDy library, although polynomials appear to be a reasonable choice for fluid 
systems, based on the quadratic nonlinearity in the Navier-Stokes equations. Given 
time series of a, and a), we thus define a library of candidate atoms 


> 2 a ae ae 
Olaa) =[1 a a a aya, ay ay aja aa, a] 


so that the unknown system can be recast as 


da 

ra = O(a, a2)ğ1> 

da (9.10) 
ae = O(a, ay)§>, 


where €, and €, are the solutions of a sparsity-promoting regression problem. After 
some cross-validation, the following system has been identified: 


d 

= = 0.094; — 0.77, — 0.016(aj + a5)a, — 0.07(a; + a5), 

a (9.11) 
ae = 0.8a, + 0.18a, + 0.06(a? + a5)a, — 0.03(a? + a3)ap. 


As for the POD-Galerkin reduced-order model derived in Section 9.3, let us first inves- 
tigate whether the identified model captures the key physics of the problem before 
discussing its accuracy. 


9.4.3 Does the model capture the key physics? 


In order to capture the key physics, the identified model (9.11) needs to fulfill the same 

conditions as those fulfilled by the POD-Galerkin reduced-order model, namely: 

Property 1: The model has a single fixed point located at a = 0. 

Property 2: The unstable subspace E” of the model linearized in the vicinity of a = 0 
is two-dimensional and associated to a complex—conjugate eigenpair. 

Property 3: As t — oo, the system eventually evolves toward a structurally stable limit 
cycle. 
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Anyone familiar with dynamical system theory might recognize that the model (9.11) 
identified with SINDy corresponds to the normal form of a supercritical Andronov- 
Poincaré—Hopf bifurcation whose phase portrait is depicted in Figure 9.13. As such, the 
identified model fulfills all three conditions at once and thus captures the key physics 
of the problem. Identifying such a normal form is consistent with earlier works on the 
same flow configuration [102, 94, 123, 77]. 


Figure 9.13: Phase plane of the low-order model identified using SINDy. The red dot indicates the 
linearly unstable fixed point while the red circle highlights the attracting limit cycle. 


Before discussing its accuracy, let us make use of the nonlinear correlations identified 
in Section 9.4.1 to recast the present model as 


d [+] 7 fee 0.19a,) -0.77(1+ | [e] 
ay 


dt a2 0.8(1 + 0.07a,) 0.18(1 — 0.18a,) 


(9.12) 
ay = 0.41 (a + a3). 


In this form, the identified model strongly underlines the nonlinear feedback mech- 
anism existing between the vortex shedding described by a, and a, and the induced 
distortion characterized by a,. It can moreover be understood as a low-dimensional 
counterpart of the self-consistent model proposed by Manti¢-Lugo et al. [68] wherein 
the “instantaneous” mean flow u is governed by 


1 —— 
V- (i1@ i) + Vp - Rev i= -V-(u'@u’), 

with u’ @ u’ being the fluctuation’s Reynolds stress tensor, while the fluctuation itself 
is governed by the Navier-Stokes equations linearized in the vicinity of the “instanta- 
neous” mean flow 

ou’ E 1 j = 1 1 2) 

——+V-(ueu +u @uU)=-Vp + —Vu. 
ot ( ) P Re 
Comparing these two models, it is quite striking that they have a similar structure and 
thus both describe the same physics. If one considers an infinitesimal perturbation u’, 
its Reynolds stresses become negligible and the instantaneous mean flow wis nothing 
but the linearly unstable base flow u,. However, as the amplitude of the fluctuation 
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grows, so do its Reynolds stresses, causing the instantaneous mean flow u to slowly 
deviate from the base flow u,. Concurrently, this distortion impacts the dynamics of 
the fluctuation through the linearized convective term V-(u@u' +u’ gü). This process 
then continues until the distortion ü- u, is such that the instantaneous growth rate of 
the fluctuation is zero (i. e., the amplitude of the fluctuation no longer grows), hence 
resulting in the marginally stable mean flow. Using the identified model, this evolution 
of the instantaneous growth rate of the instability as a function of the distortion is 
illustrated in Figure 9.14. 


0 0.5 | 
aa / max(d,) 


Figure 9.14: Evolution of the instantaneous growth rate o as a function of the distortion a,. As the 
distortion increases, the flow evolves from the linearly unstable base flow to the marginally stable 
mean flow. 


9.4.4 How accurate is it? 


Let us now assess the accuracy of the identified model compared to direct numeri- 
cal simulation. The initial velocity field used in our DNS is first projected onto the 
span of the leading POD modes. The corresponding POD coefficients a,(0) and a,(0) 
are then used as the initial condition for our reduced-order model. Figure 9.15 pro- 
vides a comparison of the trajectory of the system in the phase plane (a), ay) obtained 
from direct numerical simulation (—) and predicted by our reduced-order model (—-). 
Surprisingly, an almost perfect agreement is obtained. Note however that this is no 
overfitting. Indeed, even though the two trajectories overlap in the (a,, a,)-plane, the 
corresponding temporal evolutions slightly differ due to a small underestimation of 
the instability growth rate as discussed shortly. 
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Figure 9.15: Comparison of the evolution of a, and a, obtained from direct numerical simulation (-) 
and predicted by the identified low-order model (—-). 
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Since both the identified model (9.11) and the nonlinear correlations (9.8) are solely 
defined in terms of the POD coefficients, it is thus quite straightforward to reconstruct 
an estimate of the flow field as done for the POD-Galerkin reduced-order model. Fig- 
ure 9.16 depicts the evolution of the base flow-subtracted fluctuation’s kinetic energy 
as a function of time observed in direct numerical simulation as well as the evolution 
predicted by the POD-Galerkin reduced-order model derived in Section 9.3 and by the 
present combination of the manifold model and associated nonlinear correlations. 
Quite clearly, the accuracy of the model proposed in the present section largely out- 
performs that of the classical POD-Galerkin reduced-order model. In particular, our 
model does not suffer from the energy overshoot as nonlinear saturation occurs nor 
does it display the saturation to a higher energy level once the system evolves onto the 
final limit cycle. However, because we use POD modes computed from the limit cycle 
dynamics, the flow reconstructed in the vicinity of the fixed point actually differs from 
the true one since these POD modes provide only a crude approximation of the in- 
stability modes. This continuous mode deformation problem can however be solved 
using Grassmann manifold interpolation techniques discussed in the upcoming sec- 
tion. Finally, Figure 9.16 also highlights that the growth rate of the instability is slightly 
underestimated by our model, although nothing comparable to the underestimation 
of the POD-Galerkin ROM. Two different approaches can be used to correct this minor 
flaw: 

1. Instead of restricting ourselves to cubic monomials in a, and a,, one can include 
up to seventh-order monomials in the library ©(a,, a,) used for the system identi- 
fication. The resulting model then corresponds to a higher-order expansion of the 
supercritical Hopf bifurcation normal form. 

2. Alternatively, if the growth rate of the instability is known a priori, one can force 
the linearized low-dimensional operator to have the same eigenvalues as its high- 
dimensional counterpart. Such an approach then relies on constrained optimiza- 
tion techniques discussed in [61] and [63]. 


—: DNS —— SINDy ROM === POD-Galerkin ROM 


0 100 200 300 100 


Figure 9.16: Evolution as a function of time of the base flow-subtracted fluctuation’s kinetic energy 
E(t) for the DNS, the POD-Galerkin ROM derived in Section 9.3, and the model identified using 
SINDy. Note that, for the latter, the model predicts only the evolution of the a, and a, POD coeffi- 
cients. The other coefficients (a4, a3, and a,) are then reconstructed using the nonlinear correlations 
identified previously. 
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Although not discussed herein, both approaches have been tested and are illustrated 
in the accompanying Jupyter Notebook. Both of them result in a more accurate low- 
order model even though the resulting model is either more complex (i. e., includes 
higher-order terms) or requires more advanced computational techniques for the iden- 
tification (i. e., constrained £,-penalized regression). 


9.4.5 Solving the continuous mode deformation problem: 
Grassmann manifold interpolation 


The previous section highlighted how the transient and posttransient dynamics of the 
two-dimensional cylinder flow could be modeled by a simple self-exciting self limit- 
ing quasi-harmonic oscillator whose degrees of freedom correspond to the amplitudes 
a,(t) and a(t) of the two leading POD modes. If one considers only the shift mode and 
the first two pairs of POD modes computed from the limit cycle dynamics, the instan- 
taneous fluctuating velocity field u' (x, t) is then approximated by 


4 
u' (x, t) = Upog(% t) = Ug(X)aq(t) + X ualt). (9.13) 

i=1 
It must be noted, however, that while the above Galerkin expansion provides a highly 
accurate approximation of the velocity field once the flow evolves onto the limit cycle, 
it poorly approximates the fluctuation’s velocity field during the phase of exponential 
growth. This is illustrated in Figure 9.19, which depicts the instantaneous relative error 


lu’ (x, t) — Upoa (x, ÐI? 


Een) = a, OI 


As shown, the relative error for the POD reconstruction during the initial stage of tran- 
sition is of the order of 50 %. This mismatch results from the inability of the Galerkin 
expansion (9.13) to capture the continuous mode deformation taking place as the sys- 
tem evolves from the vicinity of the base flow to that of the mean flow. 

One way to circumvent this issue is to reconstruct the flow field based on the fol- 
lowing parameterized Galerkin expansion 


4 

u' (x, t) = Ug(X, t) = Uy (X, Aq)ag(t) + X u(x, ay)a;(t). (9.14) 

i=1 

In [75, 57, 103], the parameterized expansion modes were computed simply by linearly 
interpolating between the instability modes obtained from linear stability analysis 
and the POD modes from the limit cycle dynamics. Although extremely simple to im- 
plement, the elements of the resulting reduced-order basis unfortunately do not form 
in general an orthonormal set of vectors. Taking into account the fact that the insta- 
bility modes continuously deform into the POD modes as the system evolves onto the 


9 Sparse feature-based manifold modeling —— 305 


low-dimensional manifold structuring its phase space, a better reduced-order basis 
can however be obtained using so-called Grassmann manifold interpolation. Such an 
interpolation technique has been used in [3, 2] to derive linear parameterized reduced- 
order models for aeroelastic problems. Detailed mathematical derivation of the inter- 
polation scheme is beyond the scope of the present contribution and only the result- 
ing algorithmic implementation will be described hereafter. Interested readers are re- 
ferred to the PhD thesis of Amsallem [2] for more details. Note moreover that Grass- 
mann manifold interpolation is also covered in Chapter 9 of Volume 1 of the present 
book series. 

Let us consider the linearly unstable base flow and the marginally stable mean 
flow as two different operating points of the same system parameterized by the relative 


distortion s = = i a The base flow thus corresponds to Sọ = 0, while the mean flow 


corresponds to s; = 1. Furthermore, let us denote by By € R””? a basis of POD modes 
computed from the snapshots taken during the phase of exponential growth (hereafter 
denoted as weakly nonlinear POD modes, see the first row of Figure 9.18), while the 
POD basis computed from the mean flow will be denoted as ®, € R”®, Finally, let us 
introduce the Grassmann manifold of n x 5 orthonormal matrices G(n,5) and denote 
by o and ¢, the coordinates associated with our two previous bases on this manifold. 
Given ®, and ®,, our goal is thus to compute ®(s), i. e., the reduced-order basis for s € 
[0, 1], under the constraint that it has to live onto G(n, 5). A simple three-step procedure 
has been derived by [3] for that purpose: 
1. Compute the projection of ®, onto the tangent space of the Grassmann manifold 
G(n, 5) at the point o. This projection onto the tangent space is given by the so- 
called logarithmic operator at point ho 


(T - @y@})@,(@f@,) * = UEV", 


Å i (9.15) 
T=Utan (z)V , 


with T being the projection of ® onto the tangent space considered. 

2. Because this tangent space is flat, one can use simple linear interpolation to ob- 
tain T (s), i. e., the projection of the yet-unknown basis ®(s) onto the tangent space 
of the Grassmann manifold at pọ. We then have 


I(s) = U(s tan '(%))V". (9.16) 


Note that, by construction, ['(0) = 0. 
3. Finally, the projection back onto the Grassmann manifold G is computed by the 
so-called exponential operator at point họ given by 


@(s) = Do V cos(s tan '(Z)) + Usin(s tan”'(2)). (9.17) 


The overall procedure is schematically represented in Figure 9.17b. Note that, by con- 
struction, the reduced-order basis ®(s) is orthonormal and continuously varies from 
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Exp. map 
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(a) Linear interpolation (b) Grassmann manifold interpolation 


Figure 9.17: Illustration of different reduced-order basis interpolation techniques; ġo denotes our 
reference point (i. e., the weakly nonlinear POD basis) and @, corresponds to the mean flow operat- 
ing condition for which we use the classical POD modes. The parameter s is the relative amplitude 
of the distortion for which we want to interpolate the corresponding reduced-order basis $(s). The 
black thick line highlights the manifold onto which our reduced-order bases should live. 


@, for s = 0 to ®, for s = 1. This is illustrated in Figure 9.18 wherein the vorticity field 
of the instantaneous shift mode and the corresponding first and second harmonics of 
the vortex shedding are shown for various values of the relative distortion s, namely, 
s = 0, 0.25, 0.5, 0.75, and 1. Finally, Figure 9.19a depicts the evolution as a function of 
time of the relative projection error 


IT- B®" )u' (x, t)? 


Pn) = Wwe OI 


> 


where ® is either given by the classical POD basis ®, or the one obtained from Grass- 
mann manifold interpolation ®(s). Although both bases have the same cardinality, 
the one parameterized by the instantaneous relative distortion s largely outperforms 
the classical POD one in terms of reconstruction accuracy, notably during the phase 
of exponential growth. This is particularly visible in Figure 9.19b depicting the spatial 
distribution of the projection error. These results further confirm the inherent low- 
dimensionality of the problem considered despite the continuous mode deformation 
occurring as nonlinear saturation takes place. 


9.4.6 Limitations of the present approach 


Although the POD-Galerkin reduced-order model derived in Section 9.3 was able to 
capture the key physics of the problem investigated, it nonetheless suffers from a num- 
ber of major limitations listed in Section 9.3.5. On the other hand, the present sec- 
tion illustrated how one could identify a highly accurate and interpretable low-order 
model of the system by taking into account nonlinear correlations in the POD decom- 
position and the existence of a low-dimensional manifold. The existence of this low- 
dimensional manifold moreover enabled us to propose a highly accurate parameter- 
ized projection basis largely outperforming classical POD-Galerkin expansion of the 
velocity, notably in the initial stage of transition where the fluctuation’s velocity field 
is well approximated by the instability modes rather than the POD ones. 
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Figure 9.18: Evolution of the different POD modes obtained by Grassmann manifold interpolation as 
the flow evolves from the linearly unstable base flow (top) to the marginally stable mean flow (bot- 
tom). The intermediate rows correspond to a relative distortion of 25%, 50%, and 75%, respectively. 
Column (a) depicts the shift mode ua, (b) depicts the first harmonics of the vortex shedding, and (c) 
depicts the second harmonics. Note that, for each value of the relative distortion, these modes form 
an orthonormal set of vectors. 


To the best of our knowledge, the present reduced-order model is the lowest- 
dimensional and yet most accurate reduced-order model capturing the transient and 
posttransient dynamics of the two-dimensional cylinder flow. Note moreover that 
the exact same methodology is likely to be directly applicable to any other flow con- 
figuration exhibiting similar dynamics. Despite its impressive accuracy, one must 
however remain conscious that the methodology proposed herein also has some lim- 
itations. First and foremost, the identification of the reduced-order model relied on 
the existence of a low-dimensional manifold and on our ability to define a corre- 
sponding nonlinear embedding of the original high-dimensional data. Although such 
low-dimensional nonlinear manifolds are likely to exist for a large class of dissipative 
dynamical systems, they may however be higher-dimensional and/or more compli- 
cated to capture. Nonetheless, in such cases one could use advanced techniques 
from manifold learning such as kPCA [92, 93], Isomap [107], LLE and its variants 
[84, 121, 35], spectral embedding [10], MDS [17], or autoencoders [13]. 

Secondly, we assumed that the right-hand side f (a) of our low-order model 


da 
aE = f(a) 


could be expressed as a linear combination of monomials in a, and a). While this 
choice may be justified for a large class of dynamical systems, the present choice pre- 
cludes the identification of systems involving other types of nonlinearities, such as 
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Figure 9.19: (a) Comparison of the relative error for the orthogonal projection of the base flow- 
subtracted fluctuation’s velocity field onto either the leading five POD modes (-) extracted from 
the limit cycle dynamics or the Grassmann interpolated ones (——). The direct numerical simulation 
has been started from an initial condition close to the linearly unstable base flow. (b) Spatial distri- 
bution of the projection error at various times. The vertical velocity component is shown. From top 
to bottom: t = 6 (exponential growth of the instability), t = 60 (onset of nonlinear saturation), and 
t = 120 (asymptotic limit cycle). 


rational functions. It must be noted however that the SINDy framework is quite exten- 
sible and various extensions have been proposed since [22] to enable the identification 
of dynamical systems with exotic nonlinearities; see for instance [66]. Alternatively, if 
the dynamics appear to be strongly nonlinear and not expressible in terms of classi- 
cal analytical functions, one could include wavelets in the library O(a) used in the 
identification or turn to a class of neural networks known as long short-term memory 
(LSTM). Although one would sacrifice interpretability by doing so, recent works have 
shown that such LSTM deep neural networks are able to capture and reproduce the 
chaotic spatio-temporal dynamics of the Kuramoto-Sivashinky equation [112, 26]. 


9.5 Good practices 


The two-dimensional cylinder flow at Re = 100 is a prototypical example from fluid dy- 
namics capturing the key physics of bluff body flows. Despite the low-dimensionality 
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of the flow dynamics, it has been shown that a reduced-order model derived from a 

naive POD-Galerkin projection procedure fails to accurately reproduce the dynamics 

of the flow, most notably its transient dynamics. The key reasons for this failure, ex- 
plained in [77], are twofold: 

1. Galerkin projection of the Navier-Stokes equations onto the span of a low- 
dimensional POD basis causes a disruption of the energy cascade, hence giving 
rise to the energy overshoot illustrated in Figure 9.9. 

2. POD modes are classically computed from statistically steady operating condi- 
tions. Consequently, this set of modes may provide only a crude approximation 
of the fluctuation’s velocity field during transient dynamics. As a consequence, 
the corresponding low-dimensional linear operator obtained from Galerkin pro- 
jection does not correctly capture the spectral properties of its high-dimensional 
counterpart. 


Recent advances in data-driven techniques and machine learning are likely to help 
overcoming these limitations. It must be emphasized however that, despite their im- 
pressive successes regularly reported in mainstream and scientific media, blindly ap- 
plying techniques from machine learning (and in particular from deep learning) to 
fluid dynamics problems may give rise to overly complicated models. The aim of this 
section is to discuss a set of good practices that, according to the authors, are of crucial 
importance when it comes to data-driven reduced-order modeling. 


9.5.1 Dimensionality reduction 


The aim of reduced-order modeling is to obtain a low-dimensional representation of 
the dynamics of the original high-dimensional system. The very first step is thus to 
apply dimensionality reduction. POD, which is discussed at length in this book series, 
is the standard choice in mechanical engineering due to its ability to rank the modes 
according to the fraction of the fluctuation’s kinetic energy they capture. Once the 
POD modes have been computed, most of the reduced-order models proposed in the 
literature then carry on directly with the derivation of the low-dimensional model gov- 
erning the dynamics of these modes. It must be noted, however, that, as discussed in 
the previous section, POD analysis provides a set of modes whose temporal evolutions 
are only linearly uncorrelated. Hence, truncated POD corresponds simply to an opti- 
mal linear embedding of our original high-dimensional data set into a lower linear 
subspace. While this property might be beneficial for reduced-order models of linear 
systems, dissipative nonlinear dynamical systems are typically characterized by non- 
linear correlations across vastly different ranges of temporal and/or spatial scales. 
Consequently, if the data turn out to live on a low-dimensional nonlinear manifold, 
POD analysis would then overestimate the number of dimensions required to describe 
the dynamics of the system. Accounting for these nonlinear correlations is thus a key 
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problem for standard reduced-order modeling strategies which is often disregarded 
by practitioners, although it may cause the identified/derived reduced-order model to 
be unnecessarily complicated. 

Looking for nonlinear correlations between the various features of a multivariate 
time series is obviously significantly more complicated than looking for simple lin- 
ear correlations. Given the quadratic nature of the nonlinear convective term in the 
Navier-Stokes equations, it seems however reasonable to restrict ourselves to poly- 
nomial correlations. Moreover, when the investigated flow exhibits only periodic dy- 
namics as for the one considered herein, one can simply guess a priori the variables 
involved in the correlations by considering a limited number of triadic interactions. 
Polynomial regression can then be used to unravel the exact form of these nonlinear 
correlations. For more complicated flow configurations (e. g., chaotic and/or higher- 
dimensional dynamics), this task can however quickly become intractable without 
further preprocessing. Recently, Lopez-Paz et al. [64] proposed a new correlation met- 
ric to unravel whether two features of a multivariate time series are nonlinearly cor- 
related or not: the randomized dependence coefficient (RDC). Mathematical derivation 
of this metric is far beyond the scope of this contribution and interested readers are 
referred to the original paper [64] for more details. Note that this nonlinear correla- 
tion metric is extremely simple to use and can be implemented with less than 10 lines 
of R or Python. Preliminary results on a high Reynolds number shear-driven cavity 
flow have shown that the shear-layer dynamics and inner-cavity flow were only weakly 
nonlinearly correlated, thus considerably simplifying the identification of a reduced- 
order model with only four degrees of freedom. As an element of comparison, a clas- 
sical POD-Galerkin reduced-order model would involve 12 to 15 degrees of freedom. 

Although the combination of POD, RDC analysis, and polynomial regression has 
now become one of the standard approaches used by the present authors, it must be 
noted that numerous other alternatives exist to unravel nonlinear correlations. In the 
field of machine learning, these tools form a subset known as manifold learning or rep- 
resentation learning. From the authors’ point of view, a particularly interesting tech- 
nique from manifold learning is the use of so-called autoencoders. This is the subject 
of ongoing investigations by the present authors. For more details about autoencoders 
and manifold learning, please see the excellent review article by Bengio etal. [13]. 


9.5.2 System identification 


The field of system identification uses statistical methods to build mathematical 
models of dynamical systems from measured data. With respect to the classifica- 
tion proposed in [115], system identification enables us to obtain either gray-box or 
black-box models. Various methods have been proposed over the years. Some of these 
are classified in Figure 9.20 depending on the complexity (linear or nonlinear, in- 
terpretable or noninterpretable) of the resulting model. While the identification of 
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Figure 9.20: Classification of various system identification techniques based on the complexity of 
the resulting model. On the left, these techniques and their variants enable the identification of 
linear input-output models. At the center, NARMAX, EDMD, and SINDy allow one to identify inter- 
pretable input-output nonlinear dynamical systems. Finally, on the right, neural networks and their 
variants give rise to black-box strongly nonlinear models. 


a linear time-invariant dynamical system has a plethora of theoretical results, theo- 
retical guarantees for nonlinear system recovery are much more scarce. Like many 
fields, nonlinear system identification has nonetheless been revolutionized with the 
popularization of deep learning. It must be noted however that, from the authors’ 
point of view, a number of recent studies have put too much emphasis on illustrating 
deep learning techniques while discarding the possibility that the system considered 
could be modeled using a much simpler approach, notably studies which have used 
the two-dimensional cylinder flow as an illustration. Following Ockham’s razor, we 
thus strongly encourage practitioners to try linear system identification first (e. g., 
ERA, DMD, ARMAX), before moving to interpretable nonlinear system identification 
(e. g., NARMAX, SINDy) and eventually neural network-based techniques only if the 
previous two approaches have failed. 


9.6 Conclusion 


This work proposes a new reduced-order modeling procedure for unsteady fluid flows 
that yields accurate nonlinear models and insight into relevant flow structures. This 
procedure identifies sparse interpretable nonlinear models, not on the full fluid state, 
but from time-resolved measurements of the leading POD coefficients that may be 
realistically obtained in experiments. The sparsity of the model prevents overfitting 
and uncovers key nonlinear interaction terms. Although models are data-driven, they 
are interpretable, and it is also possible to incorporate partial prior knowledge of the 
physics or constraints to improve the models. If the stability modes are also available, 
it is possible to estimate the full state from the sparse model using Grassmann man- 
ifold interpolation: The full state is expanded in terms of a parameterized reduced- 
order basis, based on the dynamics. 

This methodology is illustrated using the canonical two-dimensional cylinder 
flow at Re = 100. Despite its simplicity, this flow configuration is a prototypical exam- 
ple capturing the key physics of bluff body flows. Even though this study uses data 
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from direct numerical simulations, the overall strategy is generally applicable to a 
real flow experiment with minor modifications. Despite their simplicity, the identified 
models do not suffer the same drawbacks as reduced-order models obtained from a 
Galerkin projection procedure, namely, overestimation of the duration of transients 
and energy overshoots at the onset of nonlinear saturation. Instead, the identified 
sparse models provide simple explanations for the nonlinear saturation process of 
globally unstable flows. Moreover, the models are based on sensor measurements, 
which may include POD coefficients, lift, drag, or pressure measurements that are 
physically linked to the geometry. Working in these intrinsic coordinates has the 
potential to overcome many of the limitations of classical modal-based projection 
methods, including mode deformation due to moving geometry and varying parame- 
ters. 


9.7 Perspectives 


The effectiveness of the reduced-order models identified and the modularity of the 
methodology proposed in the present work suggest a number of exciting future di- 
rections. There is significant potential for these methods to be applied broadly to ob- 
tain interpretable reduced-order models for a range of flow configurations in simu- 
lations and experiments. For example, these manifold models may be applied to de- 
velop nonlinear unsteady aerodynamic models, generalizing previous linear and lin- 
ear parameter-varying models [19, 20, 47]. 

A key motivation in this work is its extension to flow control. Given a feature vec- 
tor a and actuators characterized by a control law b(t), one could use SINDy with con- 
trol [23, 52] in order to identify low-order models 


da 
dt = f(a, b) 


that incorporate the influence of the actuation b on the dynamics of a. Combining such 
an approach with machine learning control [36] may result in interpretable models of 
entirely new flow behaviors and previously unobserved flow physics that are discov- 
ered in the controlled flow. The identified models can then serve as a low-dimensional 
representation of the actual system in order to facilitate the computation of nonlinear 
optimal feedback control laws. This is an area of active research by the authors. In the 
near future, the authors aim to apply the methodology introduced in the present work 
to the optimal control of experimental flows. 

There are a number of methodological extensions that may improve the perfor- 
mance of this sparse modeling framework. First, it will be important to demonstrate 
that these methods scale favorably to systems with higher-dimensional attractors. Be- 
cause the algorithms are based on simple regression and sparse optimization, they 
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should remain computationally tractable. Next, it may be possible to increase the ac- 
curacy of the Grassmann interpolation by building local modal libraries in different 
dynamic regimes (e. g., linear instability, saturated limit cycle, etc.). The storage re- 
quirements may further be reduced using compression techniques and sparse sam- 
pling. Finally, it has been demonstrated in [63] how such manifold models could be 
identified directly from sensor measurements such as the lift and drag coefficients. 
For the present flow configuration, the present authors identified that the dynamical 
system governing the dynamics of the lift coefficient C; (t) of the form 


&C, 2 of dC N WWedGe o 
de + (o- fac? +e( S) NE oc =o. 


Such sensor-based models are strongly related to the existence of a low-dimensional 
manifold structuring the phase space of the system investigated and to the strong 
correlations existing between the various sensor measurements considered and the 
spatio-temporal coherent structures found in the flow. Our ability to identify such 
sensor-based manifold models may eventually have a major impact in experimental 
fluid mechanics and flow control. 

A data-driven generalization of manifold models are cluster-based network mod- 
els, where the snapshots are coarse-grained by centroids and the topology is encoded 
in a transition model between these centroids [58]. Such models may approximate 
broadband-frequency wall turbulence for dozens of different wall surface actuations 
[39]. The price for for this conceptually simple, automatable, and robust reduced-order 
modeling avenue is that the manifold and sparse dynamics still need to be distilled—if 
they exist. 
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Roland Pulch 
10 Model order reduction in uncertainty 
quantification 


Abstract: Mathematical models include parameters, which are often affected by un- 
certainties due to measurement errors or imperfections of an industrial production, 
for example. In uncertainty quantification (UQ), parameter variations are often de- 
scribed by random variables or random processes. Of course the resulting stochastic 
model exhibits a higher complexity in comparison to the original model. Thus meth- 
ods of model order reduction (MOR) become attractive to save computational effort in 
UQ. We consider dynamical systems consisting of ordinary differential equations or 
differential algebraic equations. The focus is on linear dynamical systems. On the one 
hand, state variables and output variables can be expanded into a series with given 
orthogonal polynomials and unknown coefficient functions. A stochastic Galerkin 
method yields a high-dimensional deterministic system satisfied by an approxima- 
tion of the coefficient functions. A stochastic collocation method can also be written 
as a large weakly coupled deterministic system. We use traditional MOR methods 
to shrink the dimensionality of the large systems. On the other hand, quantities of 
interest typically represent probabilistic integrals like moments or failure probabil- 
ities, for example. Multidimensional quadrature methods and sampling techniques 
directly generate approximations of these statistics. Therein, the original dynami- 
cal system has to be solved many times for different realizations of the parameters. 
Thus high-dimensional dynamical systems cause a huge total computational effort. 
We discuss methods of parametric MOR to reuse a reduced-order model for differ- 
ent parameter values. Finally, numerical results are demonstrated for test examples, 
where we perform the reduction of large deterministic systems as well as parametric 
MOR. 


Keywords: Model order reduction, uncertainty quantification, polynomial chaos, 
quadrature, reduced basis method 
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10.1 Introduction 


In science and engineering, mathematical modeling often yields systems of ordinary 
differential equations (ODEs), differential algebraic equations (DAEs), or partial differ- 
ential equations (PDEs). The systems include physical parameters or geometrical pa- 
rameters, which exhibit uncertainties due to modeling errors, measurement errors, or 
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imperfections of an industrial production. Uncertainty quantification (UQ) determines 
the sensitivity of the model outputs with respect to these parameter variations. Often 
a stochastic modeling is used, where uncertain parameters are replaced by random 
variables, random processes, or spatial random fields; see [44, 46]. Now the model 
output also becomes a random process or a random field. 

The dynamical systems may be low-dimensional or high-dimensional. In any 
case, the presence of random variables increases the complexity of the differential 
equations. In some numerical methods, a stochastic discretization yields a determin- 
istic dynamical system of a much higher dimension. Now methods of model order 
reduction (MOR) are attractive to decrease the complexity and thus save computa- 
tional effort in the numerical simulation. Efficient MOR methods are already available 
for deterministic systems of differential equations or differential algebraic equations; 
see [2, 6, 7, 9, 16, 40]. 

We consider a polynomial chaos expansion (see [3, 45]) of the random quantity 
of interest (Qol) in a small- or medium-sized linear dynamical system. The expan- 
sion includes orthogonal basis polynomials and unknown time-dependent coefficient 
functions. The stochastic Galerkin method yields a coupled deterministic linear dy- 
namical system of high dimensionality, whose solution approximates the coefficient 
functions; see [21, 31, 42]. MOR methods have been applied to this high-dimensional 
system in [15, 35, 36, 38], for example. Alternatively, a stochastic collocation technique 
using a quadrature rule or a sampling method provides approximations of the coef- 
ficient functions. We write the stochastic collocation method in the form of a weakly 
coupled deterministic linear dynamical system; see [33, 34]. Now the system is high- 
dimensional and thus common MOR methods can be applied. 

The concept of parametric MOR (pMOR) becomes attractive in the case of param- 
eter-dependent systems with high dimensionality. Its aim is the efficient computation 
of reduced-order models (ROMs) for a (possibly) large number of realizations of the 
parameters. Methods of pMOR and their applications are demonstrated in [5, 8, 13], 
for example. We address the usage of pMOR in problems of UQ. An ROM represents 
a surrogate model, which can be solved instead of the original dynamical system. 
Therein, statistics of the random Qol are computed like the moments, for example. We 
also discuss reduced basis methods, which can be seen as a class of pMOR methods. 
Reduced basis techniques are efficient for many spatial problems modeled by PDEs; 
see [18, 22, 30]. A rigorous investigation of reduced basis methods for UQ can be found 
in [11]. 

This chapter is organized as follows. We introduce UQ of dynamical systems and 
the stochastic modeling in Section 10.2. MOR of deterministic dynamical systems, 
which are generated by stochastic discretizations, are addressed in Section 10.3. 
PMOR of dynamical systems with random parameters is considered in Section 10.4. 
Finally, we illustrate numerical simulations of test examples for both cases in Sec- 
tion 10.5. 


10 Model order reduction in uncertainty quantification —— 323 


10.2 Stochastic models and methods 


We review the stochastic modeling and numerical techniques to solve the arising prob- 
lems in this section. 


10.2.1 Dynamical systems 


Let a nonlinear dynamical system be given in the form 


E(u)x(t, p) = AQu)x(t, p) + F(x(t, p), p) + Bult), 
y(t, u) = C(p)x(t, 4), (10.1) 


with matrices and functions depending on physical and/or geometrical parameters 
H € D c R. The sizes of the matrices are A,E € R™", B e R™™, C e R"™*", The 
system involves a nonlinear function F : R” x D — R”. For nonsingular matrices E, 
the system consists of ODEs with the state variables x : [tọ, T] x D > R”. For singular 
matrices E, a system of DAEs is given with the inner variables x. We consider initial 
value problems 


X(to H) =Xo(M) forpu €D, (10.2) 


with a predetermined function xp : D — R”. In the case of DAEs, the initial values have 
to satisfy consistency conditions and typically depend on the physical parameters of 
the system. 

An input u : [tọ, T] > R”! is supplied to the system (10.1). An output y : [to, T] x 
D — Rt is defined as a Qol by the state variables or inner variables and the matrix C. 
Without loss of generality, we restrict the analysis to the case of a single output, i.e., 
Nout = 1. 

Efficient methods of MOR are available for linear time-invariant dynamical sys- 
tems of the form 


E(W)X(t, p) = A)x(t, p) + BDU(Ë), 
y(t, p) = C(p)x(t, p). (10.3) 


Typical MOR methods are balanced truncation, as described in Chapter 2 of Volume 1 
of Model order reduction, and Krylov subspace methods, as described in Chapter 3 of 
Volume 1 of Model order reduction. Thus we focus on linear dynamical systems (10.3). 
We assume that the linear dynamical system (10.3) is asymptotically stable for all 
H € D. Hence each eigenvalue A satisfying det(AE(u) - A(u)) = O has a strictly neg- 
ative real part. 

In MOR, a dynamical system of a much lower dimension r « n is constructed, 
whose output y is still a good approximation of the Qol y in the original system (10.1) 
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or (10.3). Let a fixed parameter value u € D be given. The ROM of the linear dynamical 
system (10.3) reads as 


Ew àit, p) = AGox(t, p) + But), 
V(t, p) = C(w)x(t, p). (10.4) 


Projection-based MOR applies two matrices V, W ¢ IR” of full rank. Typically, an 
orthogonal matrix V is supposed, i.e., V'V = I, with identity matrix I, € R””. The 
matrices of the linear dynamical system are reduced by 


Aq) = W'AG)V, Bip) = WB, 


= = (10.5) 
C(u) = CH)V, E(u) = W' EQ. 


If the two projection matrices coincide (V = W), then the MOR method is of Galerkin 
type. In common MOR, the projection matrices are computed for each required pa- 
rameter value u € D separately. pMOR addresses the parameter variation in a whole 
domain D, which is considered in Section 10.4.1. 

MOR of general nonlinear dynamical systems (10.1) represents a critical task. More 
efficient MOR methods are available for quadratic-bilinear (QB) systems; see [1, 4] and 
Chapter 3 of Volume 1 of Model order reduction. Sometimes a nonlinear dynamical sys- 
tem can be transformed into an equivalent QB system. Alternatively, the construction 
of approximative QB systems is feasible. 

Furthermore, an overview on software of MOR methods can be found in Chapter 13 
of this volume. 


10.2.2 Stochastic modeling 


The parameters are often affected by uncertainties in the systems (10.1) or (10.3). 
A common approach in UQ is to consider the parameters as independent random 
variables u : Q — D on some probability space (Q, A, P) with event space Q, sigma- 
algebra .A, and probability measure P. Often traditional probability distributions are 
applied like uniform, beta, Gaussian, etc. We assume that a joint probability density 
function p : D —> Ris available. Consequently, the output becomes a random process. 
The expected value of a measurable function f : D — R depending on the random 
variables reads as 


El] = | f(u(w)) dP(w) = [room dy, (10.6) 
P 


Q 


provided that the integral exists. The moments are E[f*] for positive integers k. The 
variance is the second central moment 


Var[f] = E[f?] - Eff] (10.7) 
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as usual and the standard deviation is its square root o[f] = Var[f]. The skewness 
and the kurtosis also represent interesting statistical quantities. They are given by the 
third and fourth standardized moments, respectively, i. e., 


_ Elf -Effly 


nj = olf! (10.8) 


forj = 3,4. 

In the dynamical systems (10.1) and (10.3), both the state variables and the output 
(QoI) change into random processes due to the stochastic modeling. Thus the com- 
plexity of the problem increases significantly. 


10.2.3 Quadrature and sampling 


In astochastic model, the desired data typically represent probabilistic integrals of the 
form (10.6). For example, moments are defined by the powers of a function and failure 
probabilities are given by an indicator function or a characteristic function; see [19]. 
Since the Qol is the random process y, the integrands are time-dependent. Sometimes 
just the Qol at a final time t = T is considered. 

A well-known approach to discretize a probabilistic integral is a multivariate 
quadrature rule or asampling method. Each technique is determined by a set of nodes 
fp, pw} c D in the parameter domain and a set of weights {y,,...,y,} c IR. The 
sum of the weights is always one. The numerical approximation of a probabilistic 
integral becomes a finite sum 


k 
[r WPW du = È yef (H°). (10.9) 
€=1 


D 


In the case of low numbers of random parameters (say, p < 5), we can use tensor 
product formulas of univariate quadrature rules. These methods become inefficient for 
higher dimensions due to the curse of dimensionality. Sparse grids or specific quadra- 
ture rules are available for large dimensions; see [17, 29, 43]. The curse of dimensional- 
ity is omitted in the sparse grid construction. A drawback is that often negative weights 
occur. However, also sparse grids become computationally infeasible for very high di- 
mensions (say, p > 30). Consequently, we have to apply sampling schemes like Monte 
Carlo or quasi-Monte Carlo methods; see [28]. The weights become y, = i for all £ 
in each sampling scheme. Pseudo-random numbers or sequences of low discrepancy 
yield the nodes in a Monte Carlo or quasi-Monte Carlo method, respectively. Any num- 
ber k can be chosen in a sampling method. Yet high-dimensional problems require 
typically a large number k to achieve sufficiently accurate results. 

Figure 10.1 illustrates examples of the nodes for different methods in the case of 
two independent uniformly distributed random variables u; € [0,1] for i = 1, 2. 
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Figure 10.1: Nodes in quadrature rules or sampling methods for two uniformly distributed random 
variables: (i) tensor product Gauss—Legendre quadrature (100 points), (ii) sparse grid of level 4 
based on the Clenshaw-Curtis rule (65 points), (iii) Monte Carlo with pseudo-random numbers 
(100 points), (iv) quasi-Monte Carlo with Halton sequence (100 points). 


Since we consider dynamical systems (10.1) or (10.3) with a Qol, a function f depends 
on y in the integrand of (10.6), i. e., f(u) = f (y(t, p)) for fixed t. Hence the evaluation 
of an approximation (10.9) requires to solve k times an initial value problem of the dy- 
namical system. This effort dominates the computation work in the stochastic model. 
The total effort is roughly proportional to k. 


10.2.4 Polynomial expansions 


The expected value (10.6) implies the inner product 


(f,g) = ELfg] = |r (H)s(wp(H) du (10.10) 
D 
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for two measurable functions depending on the random parameters. The associated 
Hilbert space is the set of square integrable functions 


L’(D,p) = {f :D > R : f measurable and E[f’] < cof. (10.11) 


Its norm reads as |[f||-2 = V(f,f). 

We apply an expansion of the random process y into a set of orthogonal poly- 
nomials. Each traditional probability distribution exhibits a sequence of orthogonal 
polynomials (see [45]): Legendre polynomials for uniform distribution, Hermite poly- 
nomials for Gaussian distribution, Jacobi polynomials for beta distribution, etc. Let 
Ce (Ug) jen, be the sequence of univariate orthonormal polynomials associated to 
the q-th random variable. The degree of the j-th polynomial is exactly j. We assume 
that the orthonormal basis is complete, which holds true for Gaussian, uniform, beta, 
and other distributions. However, there are exceptions; see [12]. The multivariate poly- 
nomials are just the products of the univariate polynomials. The set of all basis poly- 
nomials up to total degree d reads as 


(D: = PP OP’) PP Mp) th ta to tip < d} (10.12) 


There is a one-to-one mapping between the indices i = 1, 2,3,... and the multiindices 
Obj». --»jp). The basis polynomials (®;);en satisfy the orthogonality property 


O fori#k, 


ee | 1 fori=k, 


with the inner product (10.10). The cardinality of the set (10.12) is (see [46]) 


eae (p+) 


d ma (10.13) 


Hence the number of basis polynomials becomes large for large numbers p of random 
variables, even if the total degree is moderate, say, 2 < d < 5. Figure 10.2 illustrates 
the growth of the number of multivariate basis polynomials. 

Series including the orthogonal basis functions are called polynomial chaos (PC) 
expansions. The technique is analogous to Fourier series, where the trigonometric 
polynomials are just replaced by the orthonormal polynomials (10.12) with respect 
to the inner product (10.10). We expand the Qol satisfying the linear dynamical sys- 
tem (10.3) into 


y(t, W) = È wi(ÐP;: W, (10.14) 
i=1 
with a priori unknown coefficient functions w; : [to, T] —> R satisfying 


mlO = VEM, DW) = | yE DDH dp (10.15) 
D 
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Figure 10.2: Number of basis polynomials in dependence on number of random variables for differ- 
ent total degrees (in semi-logarithmic scale). 


for i € IN. We obtain a finite approximation by a truncation of the series (10.14), i. e., 


y™ (tu) = X wiOD H. (10.16) 


i=1 


The approximations (10.16) converge pointwise in time to the random process y for 
m — oo in the £-norm provided that the basis is complete. 

We include all basis polynomials (10.12) up to some total degree d, where the num- 
ber m is equal to (10.13). Let ©, = 1 be the constant polynomial of degree zero. The 
approximation (10.16) also yields the expected value as well as an approximation of 
the variance (10.7) via 


E[y(t,-)] =w,(t) and Var[y(t,-)] = > wilt). (10.17) 
i=2 


The task is to compute numerically the coefficient functions of the truncated se- 
ries (10.16) in this approach. 


10.3 MOR for stochastic expansions 


We demonstrate the potential to apply MOR methods for the numerical computation of 
the unknown coefficient functions in the PC expansions introduced in Section 10.2.4. 


10.3.1 Stochastic Galerkin method 
Let v = (vj,...,V,,)' € R™ and w = (Wj,...,Wm)' € R". The random-dependent 
system (10.3) changes into a larger coupled linear dynamical system 

Ev(t) = Av(t) + Bu(t), 

w(t) = Cv(t), (10.18) 
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with constant matrices A,E e R™*"", B e R™"n, and C e R™", Initial values 
V(to) = Vo follow from a truncated PC expansion of the initial condition (10.2). The 
stochastic Galerkin system (10.18) always features multiple outputs even if the original 
system (10.3) has a single output. The number of inputs remains the same. 

To define the matrices in the coupled system, we introduce an auxiliary matrix 
and a column vector by 


SH) = (DDD W) ij- m ER?” and s) = (D0) m €R". 
The matrices follow from the original matrices in (10.3) by probabilistic integrals 
À=E[S8A], B=Els@B], C=E[S@C], E=E[S@E], (10.19) 


using the Kronecker product ®, where the expected values (10.6) are calculated com- 
ponentwise. If the matrices of the system (10.3) represent polynomials of the random 
variables, then the expected values can often be calculated analytically. Otherwise, 
numerical quadrature schemes are required to calculate the matrices once. 

The linear Galerkin system (10.18) may be unstable, even though the systems (10.3) 
are asymptotically stable for (strictly) all realizations of the random variables. How- 
ever, such a loss of stability hardly occurs within problems from the applications. 
Examples of stability loss are just academic; cf. [37]. Thus we assume that the stochas- 
tic Galerkin system (10.18) is asymptotically stable. More details on the stochastic 
Galerkin method for linear dynamical systems can be found in [31, 32], for example. 

The dimension of the stochastic Galerkin system (10.18) is mn. This dimension- 
ality becomes huge for large numbers m given by (10.13). Thus the linear stochastic 
Galerkin system represents an excellent candidate for an MOR. Projection-based MOR 
operates on the constant matrices A, Ê, Ĉ, Ê like in (10.5). Krylov subspace methods 
were successfully applied in [23, 47]. Balanced truncation was used in [15, 33, 35]. The 
reduction is often efficient such that reduced dimensions r < mare still sufficiently 
accurate, i.e., the state space dimension is lower than the number of outputs. 


10.3.2 Stochastic collocation techniques 


If the stochastic model is solved approximately using the solutions of the dynamical 
system (10.3) for a finite number of realizations of the random parameters, then the 
approach is called a stochastic collocation method. In this context, we use a quadra- 
ture rule or a sampling scheme introduced in Section 10.2.3 to compute the unknown 
coefficient functions (10.15) of the PC expansion. 

The original dynamical systems (10.3) may be small- or medium-sized. To make 
the stochastic collocation method accessible to MOR, we construct a large auxiliary 
system; see [33, 34]. A construction of this type was also applied to It6 differential 
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equations for another purpose in [27]. Given the nodes of a quadrature rule or sam- 
pling scheme, the initial value problems 


E(u k(t, pO) = AM) x(t) + BOJU, xlo) = xo (HP), 
y(t we) = C(u)x(t, u”) (10.20) 


are solved separately for £ = 1,...,k. The integrals in (10.15) change into the finite sums 


w,(t) = 3 vD” )y ES YeD; (pO CHO xtp) (10.2) 


fori =1,2,...,m 
The systems (10.20) are collected in a single system as done in [33]. Let 


X(t) := (x(t,u)',....x(t,u)')' eR and 
w(t) := (WB), -<s Wml) 


The systems (10.20) for £ = 1,...,k together with the outputs (10.21) fori = 1,...,m 
yield the larger weakly coupled system of the form (10.18). This system consists of k 
separate subsystems (10.20), which are coupled only by supplying of the same input 
and the definition of the outputs (10.21). Thus the matrices A,E e R™®™ are block- 
diagonal. More precisely, we have 


Gu?) B®) 
Ô = =, forG € {A,E}and B= 
cu) By) 
Obviously, the weakly coupled system is asymptotically stable provided that the orig- 


inal systems (10.3) are asymptotically stable for all u € D. Likewise, we define an 
auxiliary matrix 


cu) 
€ Ren 


M 
Il 


Cu”) 
The quadrature rule (10.21) determines the output matrix Ê e RM by 
C=FC withF = (fe) € R®, fe = yd; (ph). 


Again the outputs w of (10.18) yield an approximation (10.16) of the Qol. For large num- 
bers k of nodes or samples, the dimension kn becomes huge. Now we can apply meth- 
ods of MOR to the weakly coupled system. 
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10.4 Parametric MOR for quadrature and sampling 


We show a potential to compute statistics, where ROMs from pMOR are sampled in- 
stead of the full-order models (FOMs). Hence the ROM is used as a surrogate model. 
This approach is applicable to both linear dynamical systems (10.3) and nonlinear dy- 
namical systems (10.1). The previous works [5, 8] represent surveys on pMOR. In [38], 
UQ and pMOR have already been combined in the case of linear dynamical systems. 
A specific pMOR method is presented for general dynamical systems in Chapter 7 of 
Volume 1 of Model order reduction. 


10.4.1 Application of pMOR 


Now the original parameter-dependent dynamical system is assumed to be high- 
dimensional. The aim of pMOR is to preserve the parameters in a reduction of the 
systems (10.1) or (10.3). Thus the ROMs are constructed in dependence on the param- 
eters within an offline phase, where the computation work is significant. Whenever 
an ROM is required for a particular realization of the parameters, a cheap formula is 
available within an online phase. 
In projection-based pMOR, there are mainly two possibilities to determine the pro- 
jection matrices: 
1. Parameter-dependent projections: A priori calculations yield formulas for the pro- 
jection matrices V(u), W(u) € IR”, which can be evaluated for any u € D. The 
reduced matrices become 


Aq) = Wan AG@VG), By) = WWB), 
Cw) = CQ) VM), Eq) = Ww) EGOV H), 


for varying parameters u € D. For example, a local reduced basis is computed for 
each element in a predetermined finite set of parameters. If the projection matrices 
are required for a particular u € D, then a kind of interpolation produces V (u) and 
Wp) using neighboring local bases. 

2. Constant projections: The information of the whole parameter domain D ora large 
finite set of samples is used to construct constant projection matrices Vo, Wo. Thus 
we have 


(10.22) 


V(u)=Vọ and W(u)=W, forallueD (10.23) 


in (10.22). This approach yields global projection matrices, which can be used for 
any realization of the parameters. However, a larger reduced dimension is often 
necessary for a sufficiently accurate MOR in comparison to a local construction. 


In both cases, the two projection matrices are often identically selected (W = V) and 
thus just one projection matrix has to be identified. 
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In the two variants of projection-based pMOR, the crucial computation work takes 
place in an offline phase. Often this offline phase consists in evaluations of the FOM 
for a finite set of parameter values. Such evaluations could also be used in a quadra- 
ture method applied to the FOM without an MOR. Thus the critical issue is if the pMOR 
is able to identify a sufficiently accurate global ROM still with a low computational ef- 
fort. Otherwise, the FOM could be sampled as well. This problem of certified accuracy 
in pMOR has also been recognized by [11]. If the error of the pMOR does not exceed the 
magnitude of the error in the time integration, then the described approach is reason- 
able. 

A stochastic Galerkin method can be applied to a parameter-dependent ROM; 
see [38]. This approach features both advantages and disadvantages in comparison 
to the strategy from Section 10.3.1. Alternatively, we apply the approach of quadrature 
or sampling to the ROMs in this section. 


10.4.2 Computation of global projection matrix 


We demonstrate a technique to determine a global projection matrix from a set of local 
projection matrices. Alternative strategies can be found in [5, 14, 41]. Let parameter 
values {u,..., pu} c D be given, which should generate a good representation of 
all parameters within D. We determine local projection matrices Vu) € R™ for 
j = 1,...,s by some MOR technique applied to the dynamical system (10.1) or (10.3). 
These local projection matrices are not required to be orthogonal. We collect all local 
bases in a large matrix 


Ù = (V(u™) Vu)... V(u)) € R”? (10.24) 


with? =7r,+1r,+---+7r, columns, assuming f < n. In [20], just an orthogonalization of 
a matrix like (10.24) is applied to define the global projection matrix. 

We decrease the dimension 7 of the global basis further by an approach also used 
in [38]. Moreover, this technique removes a (numerical) rank deficiency in the ma- 
trix (10.24) if so. The singular value decomposition (SVD) of the matrix (10.24) reads 
as 


V = USQ', (10.25) 


with orthogonal matrices U € R™", Q € R™ and a diagonal matrix S € R”? including 
the nonnegative singular values 0, > 0, > --- > 0;. In our application, just the first 
r singular values and their singular vectors have to be computed, which makes the SVD 
cheap. The singular vectors are the columns u,,...,u; of the matrix U. Depending on 
the decay of the singular values, the r dominant singular vectors are entered in the 
global basis (r < 7) 


Vo = (uw, U,,...,u,) € R™. (10.26) 
0 1> “2 r 
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Thus the global matrix V is already orthogonal (V) V, = I,). Furthermore, we sim- 
ply employ the Galerkin-type choice Wọ = Vo to define the second projection matrix 
in (10.23). 

If there were no parameter variations in the dynamical system, then the local 
projection matrices would be identical provided that the same MOR scheme is used. 
In (10.24), it follows that vu”) = Vw) € R™" for all k = 1,...,s with any ref- 
erence parameter u® € D. Now let this constant projection matrix be orthogonal. 
Consequently, the extended matrix (10.24) owns the singular values 


es fori=1,...,1%; 
: O fori=ro+1,...,Sřọ. 


(10.27) 


Ifa low amount of parameter variation is given in D, then the singular values of (10.24) 
will be close to the trivial instance (10.27). Thus the deviation of the singular values 
from the case (10.27) provides a measure of the sensitivity of the problem with respect 
to the parameter variation. 


10.4.3 Reduced basis methods 


The class of reduced basis methods represents a type of pMOR. In particular, this ap- 
proach is efficient in the case of stationary solutions of PDEs; cf. [18, 22]. For example, 
weak formulations of elliptic equations can be tackled. 

We consider a general problem 


L(x(q), H) = 0 (10.28) 


defined by a (differential) operator L : X x D — Rona function space ¥ with norm 
ll - lx. Let a unique solution x € ¥ exist for each u € D. The operator L may be a weak 
formulation of a PDE. Alternatively, L can identify a time-dependent solution of ODEs 
like a periodic steady-state response, for example. Moreover, a (spatial) discretization 
of PDEs yields operators whose solutions represent high-dimensional approximations 
to the exact solutions of an underlying problem. Typical spatial discretizations are 
finite element methods and finite difference schemes. 

Now we assume that a solution of (10.28) has to be computed many times for differ- 
ent realizations of the parameters. Thus we want to use a surrogate model that gener- 
ates cheap approximations. In reduced basis methods, a relatively small set of linearly 
independent solutions is identified, which form the subspace 


x, = span{x(w™),...,x(u™)}. (10.29) 


Given an arbitrary parameter value u, the associated solution of (10.28) is approxi- 
mated by a linear combination 


X(H) = x a (p)x(H"”), (10.30) 
jal 
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where the real coefficients a,,...,a, have to be determined in dependence on the pa- 
rameter value. Consequently, the operator (10.28) is modified into an approximation 


L,(x; (H),H) = 0 (10.31) 


with L, : ¥, x D > R, which identifies an approximation in the subspace ¥, and thus 
the required coefficients of (10.30). 
Two tasks have to be accomplished in the reduced basis approach: 
1. determination of the basis functions in (10.29), 
2. construction of the reduced-order operator (10.31) and its efficient numerical so- 
lution. 


The first task is typically achieved by a greedy algorithm. We approximate the param- 
eter domain D by a large finite set of samples Drain € D. Let an initial solution x(u®) 
be given. We compute the subspaces (10.29) recursively via X, = 4; U span{x(w 1} 
with 

po = arg max||x(q) - xj Wz (10.32) 

HED irain 

including the solutions of the operators (10.28) and (10.31). However, the computation 
of the solutions and the norm of their difference is often too costly in (10.32). Hence 
we replace the norm by a residual-based estimator R. The bounds 


cRy) < |x) -x7 Wl < cRy) forpu €D 


with constants c,,c) > O are required for a certified error estimation. The computa- 
tional effort of an evaluation of the residual-based criterion is low. 

The second task consists in the derivation of the approximation (10.31) to the oper- 
ator (10.28). This construction is problem-dependent. In weak formulations of PDEs, 
the original function space is just restricted to the low-dimensional subspace (10.29). 
In the case of linear operators, we compute the involved matrices a priori in the offline 
phase. The solution of (10.31) becomes cheap in the online phase now. In the case 
of nonlinear operators, a straightforward approximation still includes the complete 
nonlinear terms of (10.28). Thus we require cheap approximations of the nonlinear- 
ities. The (discrete) empirical interpolation method represents such an approximate 
construction; see [10] and the references therein. 

The efficiency of reduced basis methods can be motivated by the manifold of the 
parametric solutions 


M={XH):pE DE CX. (10.33) 


If the dependence of the PDE solutions on the parameters is (sufficiently) smooth, then 
the Kolmogorov width of the manifold is small. Consequently, a sufficiently accurate 
approximation is possible by a low-dimensional subspace. 
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Stochastic reduced basis methods were already investigated in [26, 39]. The use of 
reduced basis methods for UQ of weak formulations of PDEs was presented in detail 
by [11]. The reduced basis approach was proven to be efficient for stationary prob- 
lems in a spatial domain. Also parabolic equations which depend on time as well as 
space can be treated by these methods; see [11, 30]. However, the applicability to tran- 
sient problems like our dynamical systems (10.1) and (10.3) still has to be examined. 
We think about dynamical systems which do not result from a spatial discretization 
of a PDE. In this case, the function space 7 may represent the periodic steady-state 
response in a time interval [0, T], since periodic solutions satisfy a boundary value 
problem. The efficiency is still undecided for initial value problems. If just the solu- 
tion’s value in R” at a final time t = T represents the Qol, then reduced basis methods 
are unnecessarily complex for this task. 


10.5 Numerical examples 


We demonstrate the application of the approaches from Section 10.3 and Section 10.4 
now. 


10.5.1 MOR for stochastic expansions 


Figure 10.3 depicts the diagram of a band pass filter. The mathematical modeling 
yields an explicit system of ODEs with dimension n = 10 for five node voltages and 
five branch currents. Physical parameters are included by five capacitances, five in- 
ductances, and four resistances (p = 14). A single input voltage is supplied, whereas a 
single output voltage drops at a load resistance. The Bode plot of the linear dynamical 
system is shown for a constant choice of the parameters by Figure 10.4. We recognize 
that there is just a small frequency interval around w = 1, where the magnitude of 
oscillations remains the same, while other frequencies are damped strongly. 


U.O =z = | Uou 


Figure 10.3: Electric circuit of a band pass filter. 
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Figure 10.4: Bode plot of band pass filter for deterministic parameters. 


We replace all physical parameters by independent random variables with uniform 
probability distributions, which vary 20 % around their mean values given by the con- 
stant choice of parameters from above. Hence the PC expansion (10.14) involves the 
multivariate Legendre polynomials. In the truncated PC expansion (10.16), we include 
all basis polynomials up to total degree d = 3, which implies m = 680 basis functions 
due to (10.13). 

Now the two approaches from Section 10.3 are examined. On the one hand, we ar- 
range the stochastic Galerkin system of dimension mn = 6800 as in Section 10.3.1. The 
system matrices (10.19) are computed by a sparse grid quadrature of level 3 based on 
the Clenshaw-Curtis rule, where k = 4117 nodes arise. On the other hand, we generate 
a stochastic collocation system as in Section 10.3.2, with a sparse grid quadrature of 
level 2 of the same type with k = 421 nodes. The dimension of this weakly coupled 
system is kn = 4210. Both systems feature a single input and m outputs. The outputs 
reproduce the expected value as well as the variance of the output voltage via (10.17). 

We perform an MOR of both linear dynamical systems using the balanced trunca- 
tion technique; see [2]. A direct linear algebra algorithm yields the Cholesky factors of 
the Gramian matrices. An SVD produces the Hankel singular values in each approach, 
which are shown in Figure 10.5. The singular values exhibit a similar rate of decay in 
both linear dynamical systems. We expect a high potential for an MOR due to the fast 
decay. The singular values and singular vectors allow for the construction of ROMs 
with any dimension. 

We also perform a transient simulation to compare the accuracy of the stochastic 
expansion methods and their ROMs. The input voltage is chosen as the chirp signal 


u(t) = sin(2mt?). 


The output voltage represents the random Qol. The time interval [0, 100] is considered 
with initial values (10.2) identical to zero. In the time integration, we use the trape- 
zoidal rule with constant step size At = 0.005 in all cases. A reference solution of 
the expected value as well as the variance is computed using directly the sparse grid 
quadrature of level 3 with k = 4117 nodes (without projection to a PC expansion). 
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Figure 10.5: Dominating Hankel singular values of linear dynamical systems from the stochastic 
Galerkin method and the stochastic collocation technique in the band pass filter example. 


Hence the error of the time integration becomes negligible in comparison to the er- 
ror of the stochastic discretizations and the error of an MOR for moderate reduced di- 
mensions. Figure 10.6 shows the approximations for the expected value as well as the 
standard deviation in the FOMs. The approximations of the expected value coincide in 
all techniques. The approximations of the standard deviation agree for the stochastic 
Galerkin method. The stochastic collocation yields a slightly different approximation, 
which still captures the main dynamics. Now we consider the ROMs in the stochastic 
Galerkin method and the stochastic collocation. Table 10.1 illustrates the differences of 
the expected value as well as the variance with respect to the FOM solution for varying 
reduced dimensions. Obviously, the differences diminish for increasing dimensions, 
which confirms the quality of the used MOR. 


0.1 0.04 
— Galerkin = — Galerkin 
o — collocation 2 —-collocation 
= 0.05 — reference T 0-03 — reference 
SS > 
5 © 
a <0 p 
O = 
D oS 
Q Ẹ 
æ -0.05 a] 
D 
-0.1 : 
0 50 100 0 50 100 


time time 


Figure 10.6: Transient simulation of expected value (left) and standard deviation (right) for random 
output voltage from the stochastic Galerkin approach, the stochastic collocation scheme, and refer- 
ence solution. 


In this example, we reproduced the expected value and the variance. Nevertheless, 
more sophisticated quantities can be derived from the PC approximation (10.16) using 
the coefficient functions. 
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Table 10.1: Maximum differences (rounded to one digit) in moments between FOM and ROM for the 
stochastic Galerkin method and the stochastic collocation technique. 


Reduced dimension 10 20 30 
Galerkin Expected value 8-107 7-104 2-104 
Variance 1-104 2-10° 1-1076 
Collocation Expected value 2-107? 3:107? 4.107 
Variance 3-104 2-10° 7-10 


10.5.2 PMOR for statistics 


The anemometer system represents a benchmark in MOR; see [24, 25]. The convection- 
diffusion PDE 


pace = V-(KVT) —pagc(v- VT) +q 


models the time evolution of the temperature T with fluid density pp, thermal conduc- 
tivity x, specific heat c, and the velocity profile v. The heat flow q becomes the input. 
The output is defined as the temperature difference between two sensors. We obtain a 
rough estimate of the flow velocity v (as a part of v) by this difference. A finite element 
method performs a spatial discretization, which generates an implicit system of linear 
ODEs (10.3) with dimension n = 29008 and single-input-single-output. We arrange a 
constant fluid density pp = 1. The system still depends on the three positive parame- 
ters UW, = C, Wy = V, M3 = K. Figure 10.7 depicts the Bode plot of the linear dynamical 
system in the case of deterministic parameters 4; = 1 for i = 1,2, 3. 
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Figure 10.7: Bode plot of anemometer model for deterministic parameters. 


In the stochastic modeling, we choose independent beta distributions for each pa- 
rameter. Given a single beta-distributed random variable u € [-1,1], the probability 
density function reads as 


pu) = C(a, B) —-w2(1 + wF, (10.34) 
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with exponents a,$ > -1 and a constant C(a,f) > O for normalization. A bijective 
linear transformation maps the interval [-1, 1] to any interval [Lnins Umax]. We consider 
two choices of the parameter domain D: 

(i) small variation: p, € [0.9,1.0], 4> € [1.0, 1.1], 3 € [1.0, 1.1], 

(ii) large variation: } € [0.5, 1.0], 4> € [0.7, 1.2], 43 € [1.0, 1.5]. 


Furthermore, we select the exponents a; = 1 and £; = 3 for all i = 1, 2,3. 

In a pMOR, we use the technique from Section 10.4.2. We choose all vertices of the 
cube D for the computation of local reduced bases. Hence s = 8 parameter samples 
are involved. The one-sided Arnoldi method (see [2]) represents a moment-matching 
method, where a single expansion point 0 € C is applied in the frequency domain. 
We employ the real expansion point @ = 10“ for each parameter sample, which causes 
real-valued results. A local orthonormal basis of dimension r; = 25 is generated for 
each parameter sample j = 1,...,s. The extended matrix (10.24) consists of ? = 200 
columns. We compute its SVD (10.25) in the two cases (i) and (ii) of the parameter 
domains. The singular values are depicted in Figure 10.8. As expected, the singular 
values behave similar to the limit (10.27) of vanishing parameter dependence in the 
case (i) of small variations. The rate of decay becomes slower in the case (ii) of larger 
variations. Nowa global reduced basis (10.26) can be arranged for any dimensionr < f, 
where the singular vectors associated to the dominant singular values are included. 
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Figure 10.8: Singular values of extended matrix (10.24) in pMOR for the two choices (i) and (ii) of the 
parameter domain in the anemometer example. 


We perform a transient simulation for a comparison of the FOM and the ROM in the 
case (ii). Using the time interval [0, 0.01], the harmonic oscillation 


. [20 
u(t)=A sin( zt) 


is supplied as input with period T = 10“ and amplitude A = 10%. Initial values 
are identical to zero. The trapezoidal rule performs a time integration with constant 
step size At = Z. Our aim is to compute statistics of the random process induced by 
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the single output. We use the Gauss—Jacobi quadrature on a tensor product grid with 
k = 4 = 64 nodes. Further tests indicate that this quadrature scheme is sufficiently 
accurate. In each node, the initial value problem is solved numerically for both the 
FOM and an ROM. 

Firstly, we select the dimension r = 75 of the global basis in the ROM. Figure 10.9 
illustrates the expected value as well as the standard deviation for both FOM and ROM. 
The approximation of the expected value is appropriate, whereas the standard devi- 
ation includes incorrect oscillations in the ROM. Secondly, we arrange the reduced 
dimension r = 125. Figure 10.10 shows the expected value and the standard deviation 
again. Now the solution from the ROM represents a good approximation to both statis- 
tical quantities. Furthermore, the skewness and the kurtosis (see (10.8)), which relate 
to the third moment and the fourth moment, respectively, are displayed in Figure 10.11. 
Although the approximations of the higher moments are less accurate in the MOR, the 
dynamics as well as the magnitude of these statistics are captured correctly. 
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Figure 10.9: Expected value (left) and standard deviation (right) of random output in anemometer 
system obtained by ROM of dimension r = 75. 
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Figure 10.10: Expected value (left) and standard deviation (right) of random output in the anemo- 
meter system obtained by the ROM of dimension r = 125. 


We note that a global projection matrix computed for a parameter domain D can be 
reused for any probability distribution in D. For example, different exponents may be 
chosen in the beta distributions (10.34). 
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Figure 10.11: Skewness (left) and kurtosis (right) of random output in the anemometer system ob- 
tained by the ROM of dimension r = 125. 


10.6 Conclusions and outlook 


We discussed two approaches for MOR of dynamical systems including random pa- 
rameters to model uncertainties. On the one hand, the stochastic Galerkin method and 
the stochastic collocation technique produce high-dimensional deterministic dynam- 
ical systems, which can be reduced by traditional MOR algorithms. On the other hand, 
pMOR is applied, where an ROM is sampled instead of the FOM within a quadrature 
scheme or (quasi-)Monte Carlo method. 

In the second approach, the computational effort for the construction of the para- 
metric ROM is critical. If this effort becomes too large, then a quadrature scheme ap- 
plied to the FOM may yield results of the same quality with lower computation work. 
Hence we require efficient pMOR methods, where error bounds or error estimates are 
available to decide the quality of an ROM. More precisely, this error should be in the 
magnitude of the error in a time integration to accept the results. 

As usual, MOR of nonlinear dynamical systems is challenging also in the field of 
UQ. However, the stochastic Galerkin method is often less efficient in the case of non- 
linear dynamical systems, since some probabilistic integrals cannot be evaluated an- 
alytically. Thus sampling methods and collocation schemes are preferred. One should 
check if a given nonlinear dynamical system can be converted into a quadratic-bilinear 
system, either equivalently or approximately. Consequently, efficient MOR methods 
are available for QB systems. Research on MOR and parametric MOR still continues 
for nonlinear dynamical systems. 
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11 Reduced-order modeling of large-scale 
network systems 


Abstract: Large-scale network systems describe a wide class of complex dynamical 
systems composed of many interacting subsystems. A large number of subsystems 
and their high-dimensional dynamics often result in highly complex topology and dy- 
namics, which pose challenges to network management and operation. This chapter 
provides an overview of reduced-order modeling techniques that are developed re- 
cently for simplifying complex dynamical networks. In the first part, clustering-based 
approaches are reviewed, which aim to reduce the network scale, i. e., find a simplified 
network with a fewer number of nodes. The second part presents structure-preserving 
methods based on generalized balanced truncation, which can reduce the dynamics 
of each subsystem. 


Keywords: Reduced-order modeling, graph clustering, balanced truncation, semi- 
stable systems, Laplacian matrix 
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11.1 Introduction 


Network systems, or multiagent systems, are a class of structured systems composed 
of multiple interacting subsystems. In real life, systems taking the form of networks 
are ubiquitous, and the study of network systems has received compelling attention 
from many disciplines; see, e. g., [61, 60, 52] for an overview. Coupled chemical oscil- 
lators, cellular and metabolic networks, interconnected physical systems, and elec- 
trical power grids are only a few examples of such systems. To capture the behaviors 
and properties of network systems, graph theory is often useful [37]. The interconnec- 
tion structure among the subsystems can be represented by a graph, in which vertices 
and edges represent the subsystems and the interactions among them, respectively. 
However, when network systems are becoming more large-scale, we have to deal with 
graphs of complex topology and nodal dynamics, which can cause great difficulty in 
transient analysis, failure detection, distributed controller design, and system simu- 
lation. From a practical point of view, it is always desirable to construct a reduced- 
order model to capture the essential behavior of the original system e. g., stability and 
passivity, frequency response, and inoput/output properties, while avoiding too ex- 
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pensive computation. In the reduction of network systems, reduced-order models are 
designed not only to capture the main input-output feature of original complex net- 
work models but also to preserve the network structure such that they are usable for 
some potential applications, e. g., distributed controller design and sensor allocation 
in dynamic networks. 

In the past few decades, a variety of theories and techniques of model reduction 
have been intensively investigated for generic dynamical systems. Techniques, includ- 
ing Krylov subspace methods (also known as moment-matching), balanced trunca- 
tion, and Hankel norm approximation [4, 3, 59, 36], provide us systematic procedures 
to generate reduced-order models that well approximate the input-output mapping 
of a high-dimensional system; see [2, 5, 6] for an overview. However, when address- 
ing the reduction of dynamical networks, the direct application of these methods may 
be not advisable, since they potentially destroy the network structure such that ob- 
tained reduced-order models could not have the network feature any more. Structure- 
preserving model reduction is crucial for the application of network systems. Taking 
into account the two aspects of the complexity of network systems, namely, large-scale 
interconnection (i.e., a large number of subsystems) and high-dimensional subsys- 
tems, two types of problems are studied in the literature towards the approximation 
of network systems in a structure-preserving manner. 

The first problem aims to simplify the underlying network topology by reducing 
the number of nodes. The mainstream methods for this problem are based on graph 
clustering, which is an unsupervised learning technique widely used in data science 
and computer graphics [45, 71]. For approximating dynamical networks, clustering- 
based methods basically follow a two-step process: The first step is to partition the 
nodes into several nonoverlapping subsets (clusters), and then all the nodes in each 
cluster are aggregated into a single node. The aggregation step can be interpreted as 
a Petrov-Galerkin approximation using a clustering-based projection; see [74, 42, 58]. 
However, how to find the “best” clustering such that the approximation error is min- 
imized still remains an open question. In [58, 46], a particular clustering, called al- 
most equitable partition, is considered, which leads to an analytic H, expression for 
the reduction error. However, finding almost equitable partitions itself is rather diffi- 
cult and computationally expensive for general graphs. Clustering can also be found 
using the QR decomposition with column pivoting on the projection matrix obtained 
by the Krylov subspace method [54]. For undirected networks with tree topology, an 
asymptotically stable edge system can be considered, which has a pair of diagonal 
generalized Gramian matrices for characterizing the importance of edges. Then, ver- 
tices linked by the less important edges are iteratively clustered [8]. The notion of 
reducibility is introduced in [42, 41, 43] to characterize the uncontrollability of clus- 
ters. Using this notion, an upper bound for the network reduction error is established, 
which can determine the clustering. The works in [11, 13] extend the notion of dissim- 
ilarity for dynamical systems, where nodal behaviors are represented by the transfer 
functions mapping from external inputs to node states, and dissimilarity between two 
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nodes are quantified by the norm of their behavior deviation. Then clustering algo- 
rithms, e. g., hierarchical clustering and K-means clustering, can be adapted to group 
nodes in such a way that nodes in the same cluster are more similar to each other 
than to those in other clusters [12, 63]. Subsequent research in [12, 22, 17, 19] shows 
that the dissimilarity-based clustering method can also be extended to second-order 
networks, controlled power networks, and directed networks. In [25, 24], a framework 
is presented on how to build a reduced-order model from a given clustering. The edge 
weights in the reduced network are parameterized so that an optimization problem is 
formulated to minimize the reduction error. 

An alternative methodology to simplify the complexity of the network structure is 
based on time scale analysis, and in particular, singular perturbation approximation; 
see some of earlier works in [73, 64, 10]. Recently, this approach has also been exten- 
sively applied to biochemical systems and electric networks [65, 1, 39, 44, 27, 32, 56, 
67]. This class of approaches relies on the fact that the nodal states of network systems 
evolve over different time scales. Removing the vertices with fast states and reconnect- 
ing the remaining vertices with slow states will generate a reduced-order model that 
retains the low frequency behavior of the original network system. This methodol- 
ogy is closely related to the so-called Kron reduction in electric networks [27, 32, 56], 
where the Schur complement of a graph Laplacian is taken that is again a Laplacian of 
a smaller-scale network. The singular perturbation approximation is capable of pre- 
serving the physical meaning of a network system. However, how to identify and sep- 
arate fast/slow states is a crucial step in this approach, and its application is highly 
dependent on specific systems. 

A network system can be simplified if the dimension of individual subsystems is 
reduced, which leads to the second research direction in reduced-order modeling of 
network systems; see, e. g., [69, 57, 23]. In this framework, the approximation is ap- 
plied to each subsystem in a way that certain properties of the overall network, such 
as synchronization and stability, are preserved. Relevant methods are developed us- 
ing generalized Gramian matrices [34] that allow for more freedom to preserve some 
desired structures than the standard Gramians. Networked nonlinear robustly syn- 
chronized Lur’e-type systems are reduced in [23], which shows that performing model 
reduction on the linear component of each nonlinear subsystem allows for preserving 
the robust synchronization property of a Lur’e network. Techniques in [43, 21] can re- 
duce the complexity of network structures and subsystem dynamics simultaneously. 
In [43], the graph structure is simplified using clustering, while subsystems are re- 
duced via some orthogonal projection. In contrast, [21] reduces graph structure and 
subsystem dynamics in a unified framework of generalized balanced truncation. Al- 
though a reduced-order system that is obtained by balanced truncation does not nec- 
essarily preserve the network structure, a set of coordinates can be found in which the 
reduced-order model has a network interpretation. 

In this chapter, we will focus on the two problems of model reduction for lin- 
ear network systems with diffusive couplings. In the aspect of simplifying network 
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topology, we only review several clustering-based methods for space reasons. For 
the reduction of subsystems, we present the generalized balanced truncation as the 
main approach to perform a synchronization-preserving model reduction. The rest of 
this chapter is organized as follows. In Section 11.2, we provide preliminaries on bal- 
anced truncation, semi-stable systems, and necessary concepts in graph theory. The 
model of diffusively coupled networks is also introduced. In Section 11.3, we present 
clustering-based model reduction methods for simplifying network topology, and in 
Section 11.4, the generalized balanced truncation approach is reviewed to reduce the 
dimension of subsystems. In Section 11.5, we glance at open problems and make some 
concluding remarks. 


Notation 

The symbols R and R, denote the set of real numbers and real positive numbers, re- 
spectively; I„ is the identity matrix of size n, and 1, represents the vector in R” of 
all ones; e; is the i-th column of I,,; the cardinality of a finite set S is denoted by |S]; 
Tr(A), im(A), ker(A) denote the trace, image, and kernel of a matrix A, respectively; 
and ||G(s)|lz,,, and ||G(s)ll,, represent the H,,.-norm and H2-norm of a transfer matrix 
G(s). 


11.2 Preliminaries 


In this section, we first briefly recapitulate the theory of balancing as a basis for the 
model reduction of linear control systems. New results for semi-stable systems and 
pseudo-Gramians are also introduced. Moreover, we review some basic concepts from 
graph theory, which are then used for the modeling of network systems. 


11.2.1 Generalized balanced truncation 


From [34, 2], we review some basic facts on model reduction by using generalized bal- 
anced truncation. Consider a linear time-invariant system 


| x = Ax + Bu, (11.1) 


y= Cx, 


with A € R™”, B € R™?, and C € R?”, whose transfer function is given by G(s) := 
C(sI, - A)"'B. Let the system (11.1) be asymptotically stable and minimal, i. e., A is 
Hurwitz, the pair (A, B) is controllable, and the pair (C, A) is observable. Note that if a 
system (11.1) is not minimal, we can always use the Kalman decomposition to remove 
the uncontrollable or unobservable states from the model (11.1). Thus, a minimal state- 
space realization can be obtained, of which the transfer function also is equal to G(s). 
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For such a system (11.1), there always exist positive definite matrices P and Q sat- 
isfying the following Lyapunov inequalities: 


AP +PA' +BB' <0, (11.2a) 
A'Q+QA+C'C<O. (11.2b) 


Any P and Qas the solutions of (11.2) are called generalized controllability and observ- 
ability Gramians of the system (11.1) [34]. When the equalities are achieved in (11.2), we 
obtain the standard controllability and observability Gramians, which become unique 
solutions of the Lyapunov equations [2]. 

Similar to the standard balancing, generalized balancing of the system (11.1) 
amounts to finding a nonsingular matrix T € R™” such that P and Q are simultane- 
ously diagonalized in the following way: 


TPT’ =T QT” =z := diag(o;, 0>,...,0,), (11.3) 


where 0; > 0) > +-+- > 0, > O are called generalized Hankel singular values (GHSVs) 
of system (11.1). Using T as a coordinate transformation, we obtain a balanced real- 
ization of system (11.1), in which the state components corresponding to the smaller 
GHSVs are relatively difficult to reach and observer and thus have less influence on 
the input-output behavior. Let the triplet (A, Ê, C) be the r-dimensional reduced-order 
model (with r « n) obtained by truncating the states with the smallest GHSVs in the 
balanced system. Then, the reduced-order model G(s) := C(sI, — A) tÊ preserves sta- 
bility and moreover, an a priori upper bound for the approximation error can be ex- 
pressed in terms of the neglected GHSVs, i.e., 


IG(s)- GS) <2 È o (11.4) 
i=r+1 


11.2.2 Semi-stable systems and pseudo-Gramians 


Semi-stability is a more general concept than asymptotic stability as it allows for mul- 
tiple zero poles in a system [9, 40]. A linear system x = Ax is semi-stable if lim, e^t 
exists for all initial conditions x(0). The following lemma provides an equivalent con- 
dition for semi-stability. 


Lemma 1. [7] A system x = Ax is semi-stable if and only if the zero eigenvalues of A are 
semi-simple (i. e., the geometric multiplicity of the zero eigenvalues coincides with the 
algebraic multiplicity), and all the other eigenvalues have negative real parts. 


Let the triplet (A,B,C) be a linear semi-stable system. The definition of semi- 
stability implies that the transfer G(s) = C(sI —A)'Bis not necessarily in the H-space, 
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and the standard controllability and observability Gramians in [2] are not well-defined 
in this case. Instead, we can define a pair of pseudo-Gramians as follows [20]: 


o0 


P= fe - J)BB' (eft - J')dt, Q= e SPICE a7) at, (11.5) 
0 0 


where 7 := limoso e“ is a constant matrix. The pseudo-Gramians P and Q in (11.5) 
are well-defined for semi-stable systems and can be viewed as a generalization of stan- 
dard Gramian matrices for asymptotically stable systems. Furthermore, the pseudo- 
Gramians can be computed as 


P=P-JPI', Q=9-J'OS, (11.6) 
where P and Q are arbitrary symmetric solution of the Lyapunov equations 


AP +PA' +(I- J)BB'(I- J')=0, 
A'O+QA+(I-7')C'CU- J) =0, 


respectively. The pseudo-Gramians lead to a characterization of the H,-norm of a 
semi-stable system. 


Theorem 1. [20] Consider a semi-stable system with the triplet (A, B, C). Then, G(s) := 
C(sI — A'B € H, if and only if CIB = O. Furthermore, if I|G(s)|l,, is well-defined, then 


G(s), = Tx(CPC™) = Tr(B" QB). (11.7) 


11.2.3 Graph theory 


The concepts in graph theory are instrumental in analyzing network systems [52]. The 
interconnection structure of a network is often characterized by a graph G that consists 
of a finite and nonempty node set V := {1,2,...,n} and an edge set € c V x V. Each 
element in € is an ordered pair of elements of V, and we say that the edge is directed 
from vertex i to vertex j if (i,j) € €. This leads to the definition of the incidence matrix 
Re R™|, 


+1 if edge (i,j) € €, 
[R]; = 4-1 if edge (j,i) € £, (11.8) 


O otherwise. 


If each edge is assigned a positive value (weight), the graph G is weighted, and a 
weighted adjacency matrix W can be defined such that wj = [W]; is positive if there 
exists a directed edge from node j to node i, i.e., (j,i) € £, and w; = O otherwise. 
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A (directed) graph G is called undirected if W is symmetric. An undirected graph G 
is called simple if G does not contain self-loops (i. e., € does not contain edges of the 
form (i,i), V i), and there exists only one undirected edge between any two distinct 
nodes. Two distinct vertices i and j are said to be neighbors if there exists an edge 
between i and j, and the set M; denotes all the neighbors of node i. 

The Laplacian matrix L € R™" of a weighted graph G is defined as 


y=] Bem Mr i= (11.9) 
7 -Wij otherwise. 


Furthermore, we can define an undirected graph Laplacian using an alternative for- 
mula: 


L=RWR', (11.10) 


where R is an incidence matrix obtained by assigning an arbitrary orientation to each 
edge of G and W := diag(w;, w3, .. ., wje), with wọ being the weight associated with the 
edge k, for each k = 1,2,..., |E]. 


Remark 1. If G is a simple undirected connected graph, the associated Laplacian ma- 
trix L has the following structural properties: 

1. L'=L>0; 

2. ker(L) = im(1); 

3. Ly < Oifi#j, andL, > 0 otherwise. 


Conversely, any real square matrix satisfying the above conditions can be interpreted 
as a Laplacian matrix that uniquely represents a simple undirected connected graph. 


11.2.4 Network systems 


In this chapter, we mainly focus on an important class of networks, namely, consensus 
networks, where subsystems are interconnected via diffusive couplings. Various appli- 
cations, including formation control of mobile vehicles, synchronization in power net- 
works, and balancing in chemical kinetics, involve the concept of consensus networks 
[66, 50, 35, 33, 53, 72, 75]. 

Here, we consider a network system in which the interconnection structure is rep- 
resented by a simple weighted undirected graph with the node set V = {1,2,...,n}. The 
dynamics of each vertex (agent) are described by 


5; : | Ži = Ax; + Bv;, 


(11.11) 
ni = CXi, 
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where x; € Rf, v; € R”, and n; € R” are the state, control input, and output of node i, 
respectively. The n subsystems are interconnected such that 


p 
miv; =- >. witi- nj) + È fiju (11.12) 
JEN; jal 

where m; € R, denotes the weight of node i. In (11.12), the first term on the left is 
referred to as diffusive coupling, where w; € IR, is the entry of the adjacency matrix 
[W]; standing for the intensity of the coupling between nodes i andj. The second term 
indicates the influence from external input uj, where the value of fj € R represents the 
amplification of u; acting on vertex i. Let F € R”? be a matrix with [F]; = fij and we 
introduce the external outputs as y; = Èj- [H lanj, with y; € R” as the i-th external 
output of the network. We then represent the network system in compact form as 


a (M @I)x = (M@A-L@BC)x + (F @B)u, (11.13) 
` L y= (HeC), i 

with joint state vector x" := [x] x] ... x,] € R", external control input u’ := 
[u] uy ... up] € RP”, and external output y = [y] y} ... ¥g] € RM; M = 


diag(m,,m,...,mM,) > 0, and L € R™" is the graph Laplacian matrix that charac- 
terizes the coupling structure among the subsystems. In many studies of undirected 
networks, the matrix M = I, is considered. 

The simplest scenario in network systems is that all the vertices are just single- 
integrators, i.e., m;X; = v; with x; € R. Then, the model of a networked single- 
integrator system can be formed by taking A = 0 and B = C = 1 in (11.13), which leads 
to 


Mx = -Lx + Fu, 
fe aoe (11.14) 


y = Hx. 


A variety of physical systems are of this form, such as mass-damper systems and 
single-species reaction networks [74]. Note that the system (11.14) is call semi-stable 
[9], since L has a simple zero eigenvalue. 

An important issue in the context of diffusively coupled networks is synchroniza- 
tion. The system Z in (11.13) achieves synchronization if, for any initial conditions, the 
zero input response of (11.13) satisfies 


jim [x;(t)-x,(t)] = 0, for allij ev. (11.15) 


Using the property of L in Remark 1, it is clear that the single-integrator network 
in (11.14) can reach synchronization. However, for the general form of (11.13), we need 
to take into account the subsystems as well. Denote by 0 = A, < A, <--- <A, the eigen- 
values of the matrix M'L in ascending order. A sufficient and necessary condition for 
the synchronization of a network consisting of agents as in (11.11) is found in, e. g., 
[50]. 
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Lemma 2. The multiagent system X in (11.13) achieves synchronization if and only if A — 
A,BC is Hurwitz, for all k € {2,3,...,n}. 


11.3 Clustering-based model reduction 


In this section, we introduce clustering-based methods that combine the Petrov- 
Galerkin projection with graph clustering. A reduced-order network model can be 
constructed by using the characteristic matrix of a graph clustering. Moreover, we 
will also briefly recap some other clustering-based methods. 

Graph clustering is an important notion in graph theory [37]. Consider a connected 
undirected graph G = (V, £). A graph clustering of G is to divide its vertex set V (with 
|V| = n) into r nonempty and disjoint subsets, denoted by C,,C),...,C,, where C; is 
called a cluster (or a cell of G). 


Definition 1. The characteristic matrix of the clustering {C}, C2, . . . , C,} is characterized 
by the binary matrix II € R”” as 


1, if vertexi € Cj, 


Il]; := 11.1 
Mly T otherwise. C16) 


Note that each vertex is assigned to a unique cluster. Therefore, each row of the 
characteristic matrix II has exactly one nonzero element, and the number of nonzero 
elements in each column of I is the number of vertices in the corresponding cluster. 
Specifically, we have 


Tlt,=1, and 15H = [Gl 1C2h..-, 1C]. (11.17) 


It is worth noting that for any given undirected graph Laplacian L, the reduced matrix 
II'LI is also a Laplacian matrix, representing an undirected graph of smaller size. 
This important property allows for a structure-preserving model reduction of network 
systems using II for the Petrov—Galerkin projection. 

Let È in (11.13) be a network system with underlying graph G of n vertices. To for- 
mulate a reduced-order network model of r dimensions, we first find a graph clustering 
that partitions the vertices of G into r clusters. Then we use the characteristic matrix 
of the clustering as a basis that projects the state space of X to a reduced subspace. 
Specifically, a reduced-order model of X can be constructed via the Petrov—Galerkin 
projection framework as 
(11.18) 


S. l (Melz=(MeA-Le@B)z+(F & B)u, 
ĵ =(H@C)z, 


where M := I'MNI e€ R®”, Ê := HLI € R”, F = I" F, and Ê = HII. The new state 
vector z" := [z] z) ... z} ] € R”, where each component z; € R° represents an esti- 
mated dynamics of all the vertices in the i-th cluster, and £ = (I1 $ I)z € R™ can be an 
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approximation of the original state x. For the single-integrator network system (11.14), 
clustering-based projection yields the reduced-order model as 


| Mz = -Lz + Fu, (11.19) 


y = Hz. 


In the reduced-order models in (11.18) and (11.19), M is a positive diagonal matrix, and 
L € R”™ is a Laplacian matrix representing a graph of a lower number of vertices. More 
preciously, M and Ê can be computed as 


5 X eciec Ll, KEL 
[M] kx = oy m, [Ll = aa ee (11.20) 
ieC, diec, [L]; k=l. 


Clearly, the reduced-order models in (11.18) and (11.19) preserve the network struc- 
ture and thus can be interpreted as simplified dynamical networks with diffusive cou- 
plings. The following example then illustrates the physical meaning of a projected 
reduced-order network model. 


Example 1. Consider a mass-damper system in Figure 11.1 (left inset), where u4, u, are 
external forces acting on the first and fourth mass blocks. Suppose that all the masses 
are identical. Then we model the network system in the form of (11.14) with 


6 -3 0 -2 -1 
-3 4 1 0 0 
M=I,, L=| 0 -1 6 -2 -3 |, F= 


O O OQ OF 
Ooroo oO 


The off-diagonal entry -[L]; represents the damping coefficient of the edge (i,j). Let 
{C1; C2, C3} = {{1, 2}, {3, 5}, {4}} be the clustering of the network, which leads to 


1 1 0 0 0 
1=/0 O 1 0 1 
0 00 1 0 


A reduced-order network model is obtained as in (11.19) with 


200 4 -2 -2 1 0 
M=|0 2 0], L=] 2 5 3 l|, F=/]0 1 
001 -2 -3 5 0 1 


It is clear that each mass in the reduced network is equal to the sums of the masses in 
the corresponding cluster. Moreover, the structure of a Laplacian matrix is retained, 
which allows for a physical interpretation of the reduced model, as shown in Fig- 
ure 11.1 (right inset). 
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Figure 11.1: An illustrative example of clustering-based model reduction for a mass—damper network 
system. 


Next, the properties of the reduced-order models in (11.18) and (11.19) are dis- 
cussed. First, it is clear that system (11.19) preserves the synchronization property. 
Moreover, the following result holds. 


Lemma 3. [11, 13] Consider the single-integrator networks in (11.14) and (11.19). The im- 
pulse responses of the two systems satisfy 


f aie 2 H1i,1,F 
jim y(t) = jim yO) = Ma (11.21) 
Denote 
S:=H(sl,+L)'F, §:= A(sl,+1)'F. (11.22) 


This lemma implies the reduction error ||S - ŝ l}, is well-defined, for any clustering II. 
For the reduced-order network system (11.18), the analysis of synchronization and 
reduction error is more complicated, since the subsystem dynamics will also be in- 
volved. Denote 


G(s) : = (H 8 C)[M @ (sI, - A) + L 8 BC](F @ B), (11.23a) 
G(s) : = (H @C)[M @ (sI; — A) +L @ BC|(F @B) (11.23b) 
as the transfer matrices of the systems (11.13) and (11.18), respectively. Generally, G(s)- 
G(s) is not guaranteed to be stable. However, a theoretical guarantee can be obtained if 


the subsystem (A, B, C) in (11.11) is passive [38], namely, there exists a positive definite 
K such that 


A'K+KA<0, and C' =BK. (11.24) 
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In this case, we have the synchronization property and bounded reduction error for 
the system (11.18). 


Theorem 2. Consider the subsystem (A, B, C) in (11.11), which is passive and minimal. 

Then the following statements hold. 

1. The original network system (11.13) achieves synchronization for any L representing 
an undirected connected graph [70, 26]. 

2. The reduced-order network system (11.18) achieves synchronization for any cluster- 
ing II [8, 13]. 

3. G(s) — G(s) € Hə, for any clustering TI [8, 13]. 


In the framework of clustering-based projection, the approximation error ||G(s) — 
ĜO), only depends on the choice of graph clustering. Thus, it is a crucial problem 
in this framework to select a suitable clustering such that the obtained reduced-order 
model (11.18) can well approximate the behavior of the original network system (11.13). 
In the following subsections, we review several cluster selection methods. 


11.3.1 Almost equitable partitions 


It is suggested in [58] to place those vertices that connect to the rest of the network 
in a similar fashion into the same cluster. This idea leads to a special class of graph 
clusterings, namely, almost equitable partitions. 


Definition 2. Let G = (V,€) be a weighted undirected graph. A graph clustering 
{C,,C>,...,C,} is called an almost equitable partition if for each p,v € {1,2,...,r} with 
H + v, we have È kec, Wik = È kec, Wik Vij € Cp, where w; denotes the (i, j)-th entry of 
the adjacency matrix of G. 


If {C}, C2... ,C,} is an almost equitable partition, its characteristic matrix II has the 
key property that im(II) is L-invariant [58], i. e., L im(II) ¢ im(II). 

Consider the single-integrator network in (11.14) with V being the vertex set. In the 
context of leader—follower networks, a subset of vertices V; ¢ V are the leaders, with 
|V,| = p, which are controlled by external inputs. Moreover, F € R”® in (11.14) is the 
binary matrix such that [F]; = 1 if vertex i is the j-th leader, and [F]; = 0 otherwise. 
Assume that the output of (11.14) is given as 


y = Hx = W2R"x, (11.25) 


where R is the incidence matrix of G and W is the edge weight matrix defined 
in (11.10). Then, the output of the reduced network model (11.19) is obtained as 
y = Ax = W2R' Tx with I being the characteristic matrix of the given almost equi- 
table partition. Using the property of the output matrices that H"H = Land H'H =1, 
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an explicit H,-error can be derived, which is characterized by the cardinalities of the 
clusters containing leaders [58, 46]. 


Theorem 3. Consider the network system (11.14) with output defined in (11.25). Let II 
be the characteristic matrix of an almost equitable partition of the underlying graph: 
{C,,Cz,...,C,}. Denote S and Š as the transfer matrices of (11.14) and (11.19), respectively. 
Then, we have 


A a p 1 
IS- lh, — ISI, - MÊ, Zi TG) ee 
IsI3,, ISI, pl-i) l 


where n = |V], p = |Y;|, and k; is the integer index such that the i-th leader is within C,,. 


In [46], a formula for the H -error is also derived by assuming a specific output 
y = Lx in (11.14). If the network (11.14) is clustered according to an almost equitable 
partition {C,,C,,...,C,}, then we have 


max; <j<q(1 — Ga) if the leaders are in different clusters, 


Is-3 = 


1 otherwise, 


with k; being the integer such that the i-th leader is within C;,. 

More results on model reduction methods based on almost equitable partitions 
can be found in [46], where network systems of the form (11.13) with symmetric sub- 
systems are also discussed. 


11.3.2 Clustering of tree networks 


If the underlying graph of the considered network model (11.13) is a tree, we can re- 
sort to the model reduction procedure proposed in [8]. Consider the network model £ 
in (11.13), where the subsystems are passive and minimal and the Laplacian matrix L 
represents an undirected tree graph 7. Note that if 7 contains n vertices, then it has 
n — 1 edges. Relevant to (11.10), an edge Laplacian is defined: 


Le = R'RW, (11.27) 


where R € R™") is the oriented incidence matrix of T and W € R)*(" is the 
edge weight matrix. It is not hard to see that L, has all eigenvalues real and positive, 
and these eigenvalues coincide to the nonzero eigenvalues of L. 

Let M = I, in (11.13), and an edge system can be defined as 


xX, = (I1 8A - La ® BC)x, + (F. 8 B)u, 
i Gna ee Ble ree?) (11.28) 


Ye = (He 8 C)Xe 
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where x, = (R' @I)x € R"*, F, = R'F, and H, = HRWL,'. A dual edge system is also 
introduced with a different realization as 


Xp = I A-L BC F B ba 
x 1 In1 ® e @ BC)xs + (Ff @ Blu (11.29) 


Ye = (Hy 8 C)xs, 


with xç = (L3! @ I)Xe, Fy = Lz Fe and H; = HRW. 

Assuming that (A, B, C) is passive and minimal, the system (11.13) achieves syn- 
chronization from Theorem 2, which means that A — A, BC is Hurwitz for all nonzero 
eigenvalues A, of graph Laplacian matrix L. This further implies that both Z, and Z; are 
asymptotically stable. Asa result, generalized controllability and observability Grami- 
ans of the edge systems (11.28) and (11.29) can be analyzed. 


Lemma 4. [8] Consider the edge systems (11.28) and (11.29) of a tree network. There exist 
matrices X > 0 and Y > O such that the following inequalities hold: 


-L.X — XL} + R'FF'R< 0, (11.30) 
-L Y — YL, + WR'H' HRW < 0. (11.31) 


Moreover, P, := X ® K™ and Q; := Y @K are a generalized controllability Gramian of £, 
in (11.28) and a generalized observability Gramian of Xs in (11.29), respectively, where K 
satisfies (11.24) for the passive subsystems. 


According to [8], the matrices X and Y can be chosen to admit a diagonal structure: 


X = diag(é,,6,...,& 1), Y = diag, nz.. -> Nn-1) (11.32) 


where the ordering én; > či}1Mi+1 is imposed. Note that X and Y imply the controllabil- 
ity and observability properties of the edges, respectively, and the value of én; can be 
viewed as an indication for the importance of the i-th edge. Following a similar rea- 
soning as balanced truncation in Section 11.2.1, removing the edges according to the 
value of é;n; is meaningful. In [8], a graph clustering procedure is presented to recur- 
sively aggregate the two vertices connected by the least important edge. Furthermore, 
an a priori upper bound on the approximation error in terms of the H -norm can be 
derived. 


Theorem 4. Consider the networked system in (11.14) with M = I,,. Assume each sub- 
system is minimal and passive, and the underlying graph is a tree. Let (11.18) be the 
r-th-order reduced network system obtained by aggregating the vertices connecting by 
the least important edges of the original network. Then, the following error bound holds: 


n-1 = 
G(s) - G(s), < (5 cfm) (11.33) 


where G(s) and G(s) are transfer matrices in (11.23), [L31] iş denotes the i-th diagonal entry 
of the matrix i; and č; and n; are the diagonal entries of X and Y in (11.32), respectively. 
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Note that the proposed method in [8] heavily relies on the assumption of tree 
topology. For networks with general topology, applying this method would be chal- 
lenging, since there may not exist edge systems as in (11.28) and (11.29), which admit 
diagonal Gramians as in (11.32). 


11.3.3 Dissimilarity-based clustering 


The methods in Section 11.3.1 and Section 11.3.2 rely either on a special graph cluster- 
ing or on a specific topology. In this section, we review a dissimilarity-based method, 
which can be performed to reduce more general network systems. Clustering of data 
points in data science is usually based on some similarity measure in terms of vector 
norms. To cluster a dynamical network, we can extend the concept of dissimilarity us- 
ing the function norms, which serves as a metric for quantifying how differently two 
distinct vertices (subsystems) behave [11, 13]. 


Definition 3. Consider a network system in (11.13) or (11.14). The dissimilarity between 
vertices i and j is defined as 


Dy = nls) -= l (11.34) 


where n;(s) := (e; ® C)[M e (sI; - A) + L 8 BC](F @ B) if (11.13) is considered and 
n;(s) := e; (SM + L)'F if (11.14) is considered. 


The transfer matrix 7;(s) is the mapping from the external control signal u to the 
output of the i-th subsystem, y;, and thus 7;(s) is interpreted as the behavior of the i-th 
vertex with respect to the external inputs. The concept of dissimilarity indicates how 
different two vertices are in terms of their behaviors. It is verified in [13] that if the net- 
work system (11.13) is synchronized, Dj in (11.34) is well-defined, and a dissimilarity 
matrix D € R”” with [D] = Dy is symmetric and with zero diagonal elements and 
nonnegative off-diagonal entries. However, it could be a formidable task to compute 
the dissimilarity between each pair of vertices in a large-scale network based on its 
definition. Next, we discuss efficient methods for computing dissimilarity D;j. 

First, we consider the single-integrator network in (11.14), which is a semi-stable 
system. Following Section 11.2.2, the pseudo-controllability Gramian of (11.14) is com- 
puted as P = JP.7", where P is an arbitrary solution of 
11'M 
1M 
We refer to [15, 20] for more details. Note that Theorem 1 implies that the transfer func- 
tion error ņ;(s) — 7;(s) is in the H-space for any nodes i and j in the network, and an 
efficient method for computing D is presented based on the pseudo-controllability 
Gramian: 


-MLP - PLM! + (I- J)M'FF'M (I -J)=0, J:= (11.35) 


Dij = ve; = e;)' P(e; = e;). (11.36) 
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Next, we consider the network system (11.13) which achieves synchronization. If 
the overall system (11.13) is semi-stable, we can still apply pseudo-Gramians to com- 
pute dissimilarity. However, the subsystems in the network may be unstable. In this 
case, we present another computation approach [13]. Denote 


-I, 2 pated ae fe 
S:= | A eR™D, st =(STMISY'STM", (11.37) 


which satisfy S1 = 0, SM1 = 0, and S'S = I„;. Let 
A:=1,,@4-S'LM'S®@BC, B=S'F@B, 


where A is Hurwitz if and only if the system (11.13) achieves synchronization. 


Theorem 5. Let the network system (11.13) achieve synchronization. Then, there exists a 
symmetric matrix P € R&YV&C"Ve. which is the unique solution of the Lyapunov equa- 
tion AP + PA + BB' = 0. Moreover, 


y= THY PVT) VP), (11.38) 


where Y; := (e; - e;)'MS @C. 


The definition of pairwise dissimilarity in (11.34) measures how close two subsys- 
tems behave, and aggregating vertices with similar behaviors potentially leads to a 
small approximation error. Having dissimilarity as a metric, clustering algorithms for 
static graphs in, e. g., [45, 71] can be also adopted to solve the model reduction prob- 
lem for dynamical networks. For instant, a hierarchical clustering algorithm is applied 
in [12] as in Algorithm 11.1. 

An iterative approach for single-integrator networks can be found in [11], and 
an alternative clustering method is presented in [63], which takes into account the 
connectedness of vertices such that the vertices in each cluster form a connected 
graph. 

In Algorithm 11.1, the proximity of two clusters C, and C, is evaluated by (11.39), 
which means the average dissimilarity of the vertices in the two clusters. Other met- 
rics of cluster proximity can be used as well. For instance, we can take the smallest 
dissimilarity of the vertices from two clusters, or the largest dissimilarity of the nodes 
from two clusters. The proximity of two clusters allows us to link pairs of clusters with 
smaller proximity and place them into binary clusters. Then, the newly formed clus- 
ters can be grouped into larger ones according to the cluster proximity. In each loop, 
two clusters with the lowest proximity are merged together, and finally a binary hi- 
erarchical tree, called dendrogram, that visualizes this process can be generated; see 
Figure 11.2 in the following example. 
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Algorithm 11.1 Hierarchical clustering algorithm. 


1: Compute the dissimilarity matrix D. 
2: Place each node into a singleton cluster, i. e., C; — {i}, Vl<i<n. 
3: Find two clusters C% and G such that 


(k,l) := arg min( aay $ Y Dy). (11.39) 


ieC, jEC; 


4: Merge clusters C and G into a single cluster. 
5: Repeat steps 3 and 4 until r clusters are obtained. 
6: Compute the characteristic matrix II € R”’ and return 


M — O'MI, Î -—I'L,F CI'F. 


bad 
w 


Dissimilarity 
o p 
a N 


4 5 1 2 3 
Vertex index 


Figure 11.2: Dendrogram illustrating the hierarchical clustering of the networked mass—damper sys- 
tem. The horizontal axis is labeled by vertex numberings, while the vertical axis represents the dis- 
similarity of clusters. The dissimilarity is measured in the #,-norm, and the level at which branches 
merge indicates the dissimilarity between two clusters. 


Example 2. Consider the networked mass—damper system in Example 1. The dissim- 
ilarity matrix can be computed using either (11.36) or (11.38), which yields 


o 0.2494 0.3154 0.3919 0.4142 
0.2494 0) 0.2119 0.3688 0.3842 
D= | 0.3154 0.2119 0) 0.2410 0.2394 
0.3919 0.3688 0.2410 O 0.0396 
0.4142 0.3842 0.2394 0.0396 0) 


The minimal value is 0.0396, indicating that vertices 4 and 5 have the most similar 
behavior compared to the other pairs of vertices. Thus, vertices 4 and 5 are first ag- 
gregated, which leads to clusters: {{1}, {2}, {3}, {4, 5}}. In the hierarchical clustering, we 
check the proximities of the clusters by (11.39) and then obtain a coarser clustering 
{{1}, {2, 3}, {4, 5}}. This process can be continued until we have generated a dendrogram 
as depicted in Figure 11.2. 
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Algorithm 11.1 is based on pairwise dissimilarities of the vertices and minimizes 
within-cluster variances. The variance within a cluster can be characterized by the 
largest dissimilarity between all pairs of vertices within the cluster, which leads to an 
upper bound on the #,-approximation error [13]. 


Theorem 6. Consider the network system (11.13) with the output matrix H = I,. Let 
{C,,Cz,...,C,} be the graph clustering of the network, and let G(s) and G(s) denote the 
transfer matrices defined in (11.23). If A in (11.11) satisfies A+ A' < 0, then we have 


F 
CORGO MESA 2 Cyl - max Dj, (11.40) 
=1 


where y € R, only depends on the original system (11.13) and satisfies 


I@(A' +A)-L@(C'B'+BC) L@BC -Ie@c' 
Lec'B' -yI I <0. (11.41) 
-Iec I -yI 


If the considered network system is in the form of (11.14), we further obtain an 
error bound based on the pseudo-controllability Gramian. 


Proposition 1. [20] Let S and S in (11.22) be the transfer matrices of (11.14) and (11.19), 
respectively. We have 


IS — Sg, < ys Tr(I - HN") - ma"), (11.42) 


where IÙ = (N MIDI" M and? is the pseudo-controllability Gramian of (11.14). The 
constant y, € R, is a solution of 


MLM!+M"LM ML (I- JH" 
LM -y;I H" <0, (11.43) 
H(I- J) H -yI 


with J defined in (11.35). 


The core step in dissimilarity-based clustering is to properly define the dissimilar- 
ity of dynamical vertices. For linear time-variant networks, nodal dissimilarity can be 
always defined as the transfer from the external inputs to the vertex states. This mech- 
anism of dissimilarity-based clustering is applicable to different types of dynamical 
networks; see, e. g., [12, 19, 19, 17] for more results on second-order networks, directed 
networks, and controlled power networks. For nonlinear networks, DC gain, a func- 
tion of input amplitude, can be considered [48], in which model reduction aggregates 
state variables having similar DC gains. 
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11.3.4 Edge weighting approach 


Generally, all the existing clustering-based reduction methods fall into the framework 
of Petrov-Galerkin projections. In [25, 24], an #-optimal approach is presented, 
which does not aim to find a suitable graph clustering. Instead, this approach fo- 
cuses on how to construct a “good” reduced-order model for a given clustering. To 
formulate this problem, the topology of a reduced network can be obtained from the 
given clustering, while all the edge weights are free parameters to be determined via 
optimization algorithms. 

Consider the original network system in (11.14) with graph G. Let {C,,C,,...,C,} be 
a given graph clustering of G. Then, a quotient graph G is an r-vertex directed graph ob- 
tained by aggregating all the vertices in each cluster as a single vertex, while retaining 
connections between clusters and ignoring the edges within each cluster. If there is an 
edge (i,j) € G with vertices i,j in the same cluster, then this edge will be ignored in Ê. 
However, if the edge (i,j) satisfies i € C, andj € C), then there will be an edge (k, 1) in G. 
The incidence matrix R of the quotient graph G can be obtained by removing all the 
zero columns of II" R, where R is the incidence matrix of G, and II is the characteristic 
matrix of the clustering. Denote 


Ŵ =diag(w), withw=[w, Ww. --- wd (11.44) 
as the edge weight matrix of G, where w, € R, and x denotes the number of edges 


in G. Then, a parameterized model of a reduced-order network is obtained: 


(11.45) 


where M = II'MII, Ê = II'F, and H = HII. The edge weight matrix W is the only 
unknown to be determined. Let 


Sp i= H(sM + RWR") Ê. (11.46) 


Then, an optimization problem can be formulated to minimize the approximation er- 
ror ||S — Sp|lz,, by tuning the edge weights. Here, an example is used to demonstrate 
the parameterized modeling of a reduced network system. 


Example 3. Consider an undirected graph composed of six vertices in Figure 11.3a. 
An external force u is acting on vertex 3, and the state of vertex 4 is measured as the 
output y. Given a clustering with C, = {1,2}, C2 = {3}, C3 = {4}, C, = {5, 6}, the quotient 
graph is obtained in Figure 11.3b with the incidence matrix 


1 1 O0 0O 
r -1 0 1 0 
R= 

Oo -1 0 1 
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(a) (b) 


Figure 11.3: (a) An undirected network consisting of six vertices, in which vertex 3 is the leader and 
vertex 4 is measured. Four clusters are indicated by different colors. (b) A quotient graph obtained 
by clustering. 


Let W = diag(W,, W>, W3, W,) be the weights of the corresponding edges. The Laplacian 
matrix of the reduced network is constructed as 


w+ -ô EA 0 
awat.| 7% “tw o “Wy | 
Ww, 0 Wt, v; 
0 w, —W;  W3+W, 


and moreover, we have F = II’ F = [0 10 0]" and Ê = HII = [0010]. If in the original 
network, M = I,, in the reduced-order model (11.45), M = OMI = diag(2,1,1, 2). 


An optimization technique based on the convex-concave decomposition can be ap- 
plied to search for a set of optimal weights iteratively. Before proceeding, a necessary 
and sufficient condition for characterizing ||G,(s)||7,, is shown. 


Theorem 7. Given the network system (11.14). A reduced-order model in (11.45) satisfies 
IS- Solh, < Yp if and only if there exist matrices Q = Q' > 0,Z = Ê" > 0, and ô € R, 
such that Tr(Z) < yp, and 


QA+A'™Q QB, QE CATA. 0 AY 
BQ -ôr oļj+| 0 0 of<o0, (11.47) 
E'Q o o0 A, 0 -I 
Q 6c} 
` ~ |>0, (11.48) 
6c, ŻÊ 


where 


_ [-StIM'S, 0] . [0 -S*RWR'M™'S, S\F 
A= >, A= > B= 
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Lg 0 0 fe. E 
Ce = [HM!S, -Hit's], E= | | S,= | is if S, = | a ap 
I 0 1 i 


n-1 


Based on Theorem 7, the edge weighting problem is formulated as a minimization 
problem: 


min Tr(Z), s.t. (11.47) and (11.48) hold, (11.49) 
a>0, W 


where Z = 6Z. Consider the matrix-valued mapping 


DÀ, ô, W) = PÂ, ô) + p(Ñ), (11.50) 
where 
QA+A™Q QB, QE SATAY 0 AJ 
w(0,6)=| BÀ -ôr ol, ¢®)=| 0 o o 
E™Q 0 o0 Ay 0 


Then, the pair (Y, -@) is a psd-convex-concave decomposition of ® [24]. The bilinear 
matrix inequality (11.47) with the nonlinearity term A; A, can be handled using such 
a decomposition, which can linearize the optimization problem (11.49) at a stationary 
point W [31]. Rewrite p(W) in (11.50) as @(w) = p(W), with W € R* defined in (11.44). 
Given a point w), the linearized formulation of the problem (11.49) at w) is formu- 
lated as a convex problem: 


_min f(w) = Tr(Z) (11.51) 
Q>0,weR* 
Q 6c} A a ae 
s.t. f s ]>0, ber, 2=8z>0; 
ôCe Z 


PQ, ô) + o(W™) + DEW [w - W] < 0, 


where the derivative of p(w) is defined as 


Then, an algorithmic approach is presented in Algorithm 11.2 for solving the minimiza- 
tion problem in (11.49) in an iterative fashion. 

If w is initialized as the outcome of clustering-based projection methods, the ap- 
proximation accuracy obtained by the edge weighting approach will be better than the 
ones obtained by clustering-based projection after iteration. Furthermore, to solve the 
optimization problem in (11.49), we can also use a cross-iteration algorithm presented 
in [25]. 
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Algorithm 11.2 Iterative edge weighting. 


: Choose an initial vector W® € RX. 

: Set iteration step: k — 0. 

: repeat 

Solve (11.51) to obtain the optimal solution w*. 
k—k+1,andw™ —w*. 

: until Fu) -f u)| < €, with € a prefixed error tolerance. 
: Return W* < diag(w*). 


11.3.5 Other clustering-based methods 


In this section, several other model reduction schemes based on graph clustering are 
reviewed. The method in [14] formulates the clustering-based model reduction as a 
nonconvex optimization problem with mixed-binary variables II and the objective 
function to minimize the H,-norm of the approximation error. The error system be- 
tween (11.14) and (11.19) is defined, of which the controllability and the observability 
Gramians are used to derive an explicit expression for the gradient of the objective 
function. Then a projected gradient algorithm can be employed to solve this optimiza- 
tion problem with mathematical guarantees on its convergence. Related to the work in 
[58], a combination of the Krylov subspace method with graph clustering is proposed 
in [54], where a reduced basis is firstly found by the iterative rational Krylov algorithm, 
and then a graph partition is obtained by the QR decomposition with column pivoting 
on the projection matrix. An alternative graph-based model reduction method is pro- 
posed in [49], which finds a graph clustering based on the edge agreement protocol of 
a network (see the definition in [79]) and provides a greedy contraction algorithm as 
a suboptimal solution of graph clustering. The clustering and aggregation approach 
in [30, 29] is based on spectral analysis of Markov chains. The Kullback—Leibler di- 
vergence rate is employed as a metric to measure the difference between the original 
network model and its approximation. 

Clustering-based model reduction approaches are also found in the applications 
of other types of networks, i.e., network systems that do not reach consensus. In- 
stead, other network properties are emphasized. For instance, [51] proposes a reduc- 
tion method for scale-free networks, which are networks whose degree distribution 
follows a power law. They are roughly characterized by the presence of few vertices 
with a large degree (number of connections) and a large number of verities with small 
degree. The method in [51] preserves the eigenvector centrality of the adjacency matrix 
of the original network such that the obtained reduced network remains scale-free. 

Positive networks are considered in [42, 41]. A single-input bidirectional positive 
network is given in [42] as 


x=Ax+bu, xeR",ueR, (11.52) 
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where b € R? and A := -D - L, with D > O being a diagonal matrix (i. e., at least one 
diagonal entry of D should be positive and the rest of the diagonal entries are zero) 
and L > O being a Laplacian matrix representing an undirected connected graph. It 
is verified that A is negative definite, and thus the system (11.52) is asymptotically sta- 
ble. The structure of A can be interpreted as a network containing self-loops. In [42], 
a set of clusters is constructed based on the notion of cluster reducibility, which char- 
acterizes the uncontrollability of local state variables. By aggregating the reducible 
clusters, a reduced-order model is obtained that preserves the stability and positivity. 
The work in [41] extends this method to the directed case, where A in (11.52) is now 
assumed to be irreducible, Metzler, and semi-stable. In this case, the Frobenius eigen- 
vector of A is used for constructing the projections such that both semi-stability and 
positivity are preserved in the resulting reduced-order network model. In both [42] and 
[41], an upper bound on the approximation error is established using the cluster re- 
ducibility, and then a clustering scheme is proposed to select suitable clusters, aiming 
at minimizing the a posteriori bound on the reduction error. 


11.4 Balanced truncation of network systems 


Reducing the dimension of each subsystem also results in a simplification of overall 
networks. To reduce the dynamics of vertices, balanced truncation based on general- 
ized Gramian matrices is commonly used (see, e. g., [57, 23, 21]), in which preserving 
the synchronization property of the overall network is of particular interest. In this 
section, we review some recent results in the synchronization-preserving model reduc- 
tion of large-scale network systems using the classic generalized balanced truncation. 
For simplicity, we assume M = I, in (11.13) throughout this section. 


11.4.1 Model reduction of subsystems in networks 


Starting from a synchronized network system in (11.13), the aim of this subsection is 
to derive a network model with reduced-order subsystems such that synchronization 
is preserved in the reduced-order network in (11.18). 

If each subsystem in (11.11) is asymptotically stable, we might apply standard bal- 
anced truncation to reduce the dimension of the subsystem regardless of their inter- 
connection structure. However, this reduction is possible to destroy the property of 
the overall network system (11.13), e. g., stability and synchronization. To achieve syn- 
chronization preservation, [57] adopts a sufficient small gain type of condition to guar- 
antee synchronization of (11.13). 


Lemma 5. Denote by 0 = A, < A, < --- < A, the eigenvalues of the Laplacian matrix L. 
The network system (11.13) achieves synchronization if there exists a nonzero eigenvalue 
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A € {A),...,A,} such that A — ABC is Hurwitz and there exists a positive definite matrix K 
satisfying the Riccati inequality 


2 
(A -ABC)'K + K(A-ABC)+C'C + (5) KBB'K <0, (11.53) 


where ô := max{A — À», Àn - A}. 


It is worth noting that (11.53) is equivalent to the small gain condition 


|C(slp-A+ABC) "BI, < s, 


Let Km and Ky be the minimal and maximal real symmetric solutions of (11.53). Then 
Ky and K,, can be regarded as a pair of generalized Gramians of the system (A + 


ABC, êB, C). Applying the generalized balanced truncation introduced in Section 11.2.1, 
a reduced-order model (A + ABC, B, C) with A € R is obtained such that the small 


gain condition ||C(sI, - A + ABC) Bly < g is retained. Therefore, the following 
reduced-order network model preserves the synchronization property: 


F- (I @A-L@BC F @Byu, 
i Un ® SROS EIREDH (11.54) 


n = (H  Ĉ)Ẹ. 


Theorem 8. Consider a network system (11.13) that satisfies the synchronization condi- 
tion in Lemma 5. Then, the reduced-order network model in (11.54) obtained by general- 
ized balanced truncation using Ky and K,, achieves synchronization. 


Moreover, similar to (11.25), we assume a particular output y = (W2R™ @C)x. Then 
the error system between (11.13) and (11.54) is stable. We denote G(s) as the transfer 
matrix of system (11.54), and the model reduction error is upper-bounded as 


2yVAn 5 A 
i 


leo- GIy, < 
=k+1 
where g; are the GHSVs computed using K i and Km [57]. 

Inspired by the work [57] for linear networks, [18, 23] consider dynamical networks 
of diffusively interconnected nonlinear Lur’e subsystems. The robust synchronization 
of the Lur’e network can be characterized by a linear matrix inequality (LMI). Differ- 
ent from [57, 18], where the minimum and maximum solutions of the LMI are used 
as generalized Gramians, [23] suggests to only use the solution of the LMI with the 
minimal trace as a generalized controllability Gramian, while the observability coun- 
terpart is taken by the standard observability Gramian as the solution of the Lyapunov 
equation, which is less conservative than the LMI. Using such a pair, generalized bal- 
anced truncation is performed on the linear component of each Lur’e subsystem, and 
the resulting reduced-order network system is still guaranteed to have the robust syn- 
chronization property. 
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11.4.2 Simultaneously reduction of network structure and 
subsystems 


In the line of works [42, 41], a reduction method for network systems composed of 
higher-dimensional dissipative subsystems is presented in [43], where the subsystems 
are reduced via block-diagonal orthogonal projection, while the network structure is 
simplified using clustering. In [16], the balancing method, for the first time, is applied 
for reducing the interconnection structure of networks with diffusively coupled ver- 
tices, and more extensions are found in [78, 77] based on eigenvalue assignment and 
moment-matching. In [21], the idea in [16] is further developed and applied to general 
networks of the form (11.13). The proposed approach can reduce the complexity of net- 
work structures and individual agent dynamics simultaneously via a unified frame- 
work. 

Consider the network system = in (11.13), where each subsystem X; as in (11.11) is 
passive, namely, there exists a positive definite K such that (11.24) holds. Note that L is 
singular and A in (11.11) is not necessarily Hurwitz, implying that the overall system £ 
may be not asymptotically stable, and thus a direct application of balanced truncation 
to È is not feasible. The method in [21] starts off with a decomposition of X using a 
spectral decomposition of the graph Laplacian: 


L=TAT’=(T, 1] [* i ei . (11.56) 
2 


where T, = 1,/Vnand A := diag(A,,A3,...,A,), with A; denoting the nonzero eigen- 
values of L. Then, the system È can be split into two components, namely, an average 
module 


Ža = AZa + (1) F @ B)u, 
zad ae uae (11.57) 
Va = va (Hn 8 C)Zo 
with z, € Rf, and an asymptotically stable system 
Ż; = (n1 9A -A®@BC)z, + (F @ Bu, 
z:] s= (na )zs + (F ® B) aes 
Ys z (H 8 C)z,, 


where z, € R”, F = TT F, and H = HT,. The stability is guaranteed as the system 
ÈX achieves synchronization (Theorem 2). 

The model reduction procedure is as follows. First, we can apply balanced trun- 
cation to £, to generate a lower-order approximation &,. It meanwhile gives a reduced 
subsystem (A, Ê, C) resulting in a reduced-order average module Ê,. Combining Ê, with 
Ê, then formulates a reduced-order model © whose input-output behavior approxi- 
mates that of the original system X. However, at this stage, the network structure is 
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not necessarily preserved by ©. Then, it is desired to use a coordinate transformation 

to convert Ý to Ê, which restores the Laplacian structure. The whole procedure is sum- 

marized in Figure 11.4. There are two key problems here: 

1. How can one retain the subsystem structure in Ê, such that subsystem dynamics 
do not mix with the topological information? 

2. How can one recover a network interpretation in the reduced-order model  viaa 
coordinate transformation? 


Original Network X 


y y 
Average Module Ja Stable System Xs 
Balanced 
ie oe p Truncation 
? = (Â, Ê,Ĉ) g z 
Reduced Average Module $},  }—,_ Reduced Stable System 35 


| 


Reduced Model £ 


Coordinate 
Transformation 


Reduced Network S 


Figure 11.4: The model reduction scheme for networked passive systems, where the simplification of 
network structure and the reduction of subsystems are performed simultaneously. 


To resolve the first problem, we resort to the balanced truncation approach based on 
generalized Gramians. Suppose A in (11.56) has s distinct diagonal entries ordered as 
A, > Ay > +++ > As. We rewrite A as A = blkdiag(A Im, Astin,» --.Aslm,)» Where m; is 
the multiplicity of A;, and $}; m; = n — 1. Then, the following Lyapunov equation and 
inequality have solutions X and Y: 


-AX —-XA+FF' =0, (11.59a) 
-AY -YA+H'H <0, (11.59b) 
where X = X' > Oand Y := blkdiag(Y;, Y,,..., Y;), with Y; = Y; > O and Y; € R™”, 


fori = 1,2,...,s. The generalized controllability and observability Gramians of the 
stable system £, are characterized by the following theorem. 


Theorem 9. Let X > 0 be the unique solution of (11.59a), and let Y > O be a solution 
of (11.59b). Let Km > O and Ky > O be the minimum and maximum solutions of (11.24), 
respectively. Then the matrices 


X:=Xe@Ky and y:=YeK,, (11.60) 
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are a pair of generalized Gramians of the asymptotically stable system X.. Moreover, 
there exist two nonsingular matrices Tg and Tp such that T = Tg 8 Tp satisfies 


TXT” =T TYT" =%g 8 Èp. (11.61) 


Here, Xç := diag{0;, 02, .. ., On-1} and Xp := diag{tı, T2, ..., Te}, where 0, => 0, 2 --- > 
On-1 ANd T1 => T3 È --- = Ty are equal to the square roots of the eigenvalues of XY and 
Ky Km» respectively. 


The block-diagonal structure of Y will be crucial to guarantee that the reduced- 
order model, obtained by preforming balanced truncation on the basis of X and Y, 
can be interpreted as a network system again, as will be shown in Theorem 10 below. 


Remark 2. By the duality between controllability and observability, we can also use 
-AX -XA +FF' < 0 and -AY - YA + HĦ' Ē = 0 to characterize the pair X and Y for the 
balanced truncation, where now X is constrained to have a block-diagonal structure. 


Selecting the pair of Gramians in (11.60) with the Kronecker product structure is 
meaningful, since they can be simultaneously diagonalized, (i. e., balanced) using 
transformations of the form 7 = Tg ® Tp. Note that Tg and Tp are independently 
generated from (11.59) and (11.24). More precisely, Tg only balances the network struc- 
ture, or the triplet (A, F, H), while Tp only balances the agent dynamics, i. e., the triplet 
(A, B, C). Thus, the Laplacian dynamics and each subsystem (11.11) can be reduced in- 
dependently, allowing the resulting reduced-order model to preserve a network inter- 
pretation as well as the passivity of subsystems. 

Denote by (A,, Ê}, H;) and (A, Ê, C) := $; the reduced-order models of (A, F, H) and 
(A, B, C), respectively, where A, € RO~)*°-), Fe ROY, Ñi e RPO), A e ROK, 
Be RE”, and Ê e R™*, Here, k < £ andr < n. Consequently, the reduced-order 
models of the average module (11.57) and the stable system (11.58) are constructed: 


i 2 = Â2, + LaF @B)u, 

E go E (11.62a) 
Ña = Ja Hin 8 C)Zo 

2 2. = (L1 9 Â - Â, 8 ÊĈ)2, + (Ê, @ B)u, 

Ê | ig i rea ks (11.62b) 
Y, = (H, 8 C)Z,. 


Combining the reduced-order models $, and $,, a lower-dimensional approximation 
of the overall system 2 is formulated as 


5 | Z=([,@A-T 8 BC) + (F 8 B)u, (11.63) 


Ŷ = (H8 Ô), 


where 
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Here, T is not yet a Laplacian matrix, but it has only one zero eigenvalue at the origin 
and all the other eigenvalues are positive real. To restore a network interpretation in 
the reduced-order model Ž, the following theorem is provided in [21], which states that 
there exists a similarity transformation between T and an undirected graph Laplacian 
matrix. 


Theorem 10. A real square matrix I is similar to the Laplacian matrix £ associated with 
an weighted undirected connected graph if and only if T is diagonalizable and has an 
eigenvalue at O with multiplicity 1 while all the other eigenvalues are real and positive. 


By Theorem 10, we find a reduced Laplacian matrix Î which has the same spec- 
trum as T, namely, there exists a nonsingular matrix 7, such that Ê = T Th. The 
matrix 1 characterizes a reduced connected undirected graph G, which contains r ver- 
tices. Applying a coordinate transform Z = (T, ® I,)X to the system Ž in (11.63) yields a 
reduced-order network model 


~ { x=(,eA-LeBC)x+(FoB)u, 
(11.64) 
(H 8 
with F = T, 'F and H = H7,, It can be verified that the reduced-order network system 
Ê in (11.64) preserves synchronization. Moreover, denote the transfer matrices of X, Ê, 
X, Ê., Za, and Ê, by G, G, T,, T,, T,, and Î,, respectively. The approximation error can 
be analyzed as follows: 


IG- Gly, = (Ts + Ta) - Ês + Talay, 
< IIT; - Tella, + ITa - Tallos.,.> (11.65) 


in which an a priori upper bound on the reduction error of the stable system £, is given 
as 


n-1 £ r-1 £ 
ITs- Tle, <2} Voit; +2)) } om (11.66) 
i=r j=1 i=1 j=k+1 
with o; and qT; being the diagonal entries of 2, and Xp in (11.61), respectively. Denote 
by S; and $; the transfer matrices of X; and &;, respectively. If S; - $; € Ha, we obtain 


a 1 R 
ITa - Talla, < mEnE ISi- Sila- (11.67) 


In several special cases, the a priori error bounds on ||G — Glas, in (11.65) can be 
obtained. The first case is when we only reduce the dimension of the network while 
the agent dynamics are untouched as in [8]. In this case, we obtain ||T, - Talla, =0, 
which yields 


n-1 £ 


IG - Ge, = ITs - Tella, < 2È È oim; (11.68) 


i=r j=1 


11 Reduced-order modeling of large-scale network systems —— 373 


The second case is when the average module is not observable from the outputs 
of the overall system È or not controllable by the external inputs. Consider 


H1,=0, or 1,F=0. (11.69) 


Then, the approximation between X and Í is bounded by ||G - Gla... = |T; - T sll.» 
whose upper bound is given in (11.66). A special example of (11.69) can be found in 
[57, 58, 49], where the output matrix H in (11.13) is taken as in (11.25). 


11.5 Conclusions 


In this chapter, model reduction techniques for linear dynamical networks with dif- 
fusive couplings have been reviewed. There exists a vast amount of literature on this 
topic, and the reference list in this chapter is certainly not complete. For example, in 
[65, 28, 27, 32, 56], the approximation approaches are developed based on singular 
perturbation approximation and applied to reduce the complexity of chemical reac- 
tion networks and power networks. In [47], the interconnection topology is simplified 
by removing cycles in the network, and in, e. g., [10, 55, 48], preliminary results for 
reducing nonlinear dynamical networks are developed. Recently, a lot of interest is 
taken in the combination of network reduction and controller and observer designs. 
For example, [76] presents a linear quadratic Gaussian controller for large-scale dy- 
namical networks using the clustering-based reduction, and [68, 62] proposes the av- 
erage state observer based on reduced-order network models. 

Generally speaking, order reduction methods for linear network systems have 
been extensively investigated. However, the approximation of complex network con- 
taining nonlinear couplings or subsystems is still challenging, and the existing results 
on nonlinear networks are far from satisfactory. Another challenge in this area is the 
order reduction of heterogeneous networks, i.e., network systems composed of non- 
identical subsystems. 
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12 Model order reduction and digital twins 


Abstract: We are currently facing a substantial transformation of our industrial world 
and the way our economics are organized. This transformation, known as digitaliza- 
tion, is driven by the systemic integration of information technology in all kinds of 
devices, machines, and factories such that new smart networks are formed and new 
smart products have the ability to monitor, to forecast, and to control their behavior. 
One of the fundamental pillars of digitalization is simulation technology, since it en- 
ables the new intelligence layer in the form of digital twins which mirror the physical 
systems into the digital world — also named by Gartner Inc. as a top technology trend 
for 2017 and 2018. Creating such intelligence layers over several domains and life cycle 
phases requires, among other challenges, technologies for transforming and reducing 
complex simulation models. Exactly for this task a key technology is model order re- 
duction (MOR). However, MOR is not only a key technology within emerging digital 
twins but also helps to reduce simulation times in the existing everyday business of 
simulation engineers. This is especially important when for a simulation model a large 
number of evaluations are needed. Within this chapter we present use cases where 
MOR is a key enabler for the realization of digital services and the reduction of simu- 
lation times. Furthermore we outline the potential of MOR in the context of realizing 
the digital twin vision. 


Keywords: digital twin, virtual sensors, control, predictive maintenance, circuit sim- 
ulation 
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12.1 Introduction 


This chapter provides an overview of several projects which we worked on through- 
out the last years. These projects were initiated from different directions and perspec- 
tives since the authors of this chapter work in multiple departments across Siemens. 
Nevertheless, all of our projects were either part of concrete business opportunities 
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or of predevelopment activities to evaluate new business opportunities. This means 
that the goal was always to improve products, to develop new products, or to evaluate 
the potential lying in innovative business ideas. In this environment the application 
of model order reduction (MOR) was not a goal in its own. Instead the application of 
MOR was always triggered by the requirements coming from the project goals. In par- 
ticular for predevelopment projects such a goal is typically to evaluate the commercial 
benefit lying in new technologies, which in our case was MOR. 

In this chapter we start with outlining the underlying business visions of digital- 
ization and digital twins and the role of MOR within this vision. This part is followed 
by a report of our experience with productizing MOR algorithms. Finally, we report the 
content, the challenges, and the results of some of our projects. 

Throughout this chapter we try to give an insight into our work between the poles 
of business models and technological challenges, which is sometimes even the great- 
est challenge. 


12.2 Digitalization and digital twins 


Complexity in today’s industry is exploding. New production methods, miniaturiza- 
tion of electronics, novel sensor technologies, and last but not least the Internet of 
things have led to many disruptive developments implying more and more complex 
products. On the one hand, this offers unique opportunities, e. g., in terms of efficiency 
or autonomy of components, products, and complex systems. On the other hand, it 
challenges today’s design, engineering, operation, and service paradigms mostly fo- 
cusing on manual expert interaction, which can hardly, if at all, handle this enormous 
complexity. 

Digitalization changes everything everywhere. With the rise of new technology 
trends, such as AI foundations, intelligent things, cloud to edge, or immersive expe- 
riences [76], many of today’s paradigms can be expected to be disrupted. Not only in 
the consumer market, as we can clearly observe today, but also in the industrial and 
medical sectors we see disruptions as proven by first early adopters. 

Digital twins will be one key answer to these challenges; see, e. g., [24, 35, 81] for 
a broad overview from an engineering perspective. They are the next wave in simu- 
lation technologies (Figure 12.1). Digital twins integrate all (electronic) information 
and knowledge generated during the lifetime of a product, from the product defini- 
tion and ideation to the end of its life. Examples of these data range from the initial 
requirements which have led to the design of the product, the design and engineering 
data, which have been generated during virtual design, to operation data such as sen- 
sor values collected during operation. The data themselves are only a central asset, if 
it can be used to make relevant predictions providing the right level of information at 
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engineering 


Figure 12.1: Simulation is evolving from a troubleshooting tool to a key business driver in the form of 
digital twins. 


the right time. Ultimately, digital twins mirror products and systems from the real into 
the digital world and vice versa. 

From a high-level point of view, information included in digital twins can be 
split in to two categories: (i) pure data values with only little additional structure 
and knowledge associated, such as data gathered from sensors, and (ii) structured 
executable model-based data, in particular simulation models. Thus from this point 
of view digital twins bring together classical data-based schemes with model-based 
approaches such as simulation and optimization (Figure 12.2). 

Today, most model-based approaches, and in particular simulation, are domain- 
specific and mostly used during design and engineering. The core concept of the dig- 
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ital twin is to extend their usage along the complete life cycle and to deliver new ser- 
vices providing the right information at the right place in an efficient way, for exam- 
ple, digital twins supporting early system configuration during the sales process or 
optimization of operation and service concepts. This broad usage implies a number 
of requirements to modeling and simulation which diverge from its classical use in 
design and engineering: 

— Interactivity — Speed and accuracy define the value of simulation and digital 
twins. Being very accurate, today’s model and simulation approaches are ex- 
tremely time-consuming. Speeding them up, while retaining the right level of 
accuracy, is crucial for extending the use of digital twins. 

—- Reliability — Users of digital twins cannot be expected to be sophisticated experts, 
like it can be expected during the use in design and engineering. Thus any predic- 
tion by the digital twin must be fail-safe and/or provided along with confidence 
intervals such that no expertise is required to interpret the results or can be used 
autonomously, e. g., by controls. 

— Usability - Model-based and simulation tools are expert-centric today. Their re- 
sources are limited and thus the use of corresponding tools today is limited by 
the availability. Therefore, any digital twin solution must be accessible also for 
nonexperts from a usability perspective. 

— Security - Many business models based on the digital twin will require to ex- 
change digital twins between different parties. Reverse engineering must be pre- 
vented, such that no intellectual property is lost. 

-  Deployability — Digital twins will be used differently from the place where they 
have been created, e.g., on customer premises, in the cloud, on controls. Thus 
deployment must be easy to reduce barriers and efforts. 


The digital twin concept has been originally introduced in 2003 by Michael Grieves 
[41] and first put to public by NASA in 2012 [38]. Digital twins are considered so impor- 
tant to business, that they were named one of Gartner’s Top 10 Strategic Technology 
Trends for 2017 [76]. They are becoming a business imperative, covering the entire life 
cycle of an asset or process and forming the foundation for connected products and 
services. Companies that fail to respond will be left behind. For example, it is predicted 
that companies who invest in digital twin technology will see a 30 % improvement in 
cycle times of critical processes [77]. A potential market of 90 billion US dollar per year 
associated to corresponding offerings is predicted [28]. 

To realize the vision of digital twins, MOR is a key technology. Other key technolo- 
gies cover novel user interaction paradigms and devices (such as virtual, augmented, 
or mixed reality), technologies for merging data and model-based approaches, or se- 
mantic technologies to easier built-up systems of digital twins. 
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12.3 Model order reduction in the context of 
digitalization and digital twins 


The digital twin vision extends the expert-centric focus of modeling, simulation, and 
optimization technologies towards a digital assistance for everyone in day-to-day de- 
cisions. This is supported by a double exponential growth of capability in simulation 
technology. On the one hand, computational hardware is developing exponentially 
according to Moore’s law [94]. On the other hand, efficiency of simulation algorithms 
is subject to exponential growth as well [91]. With this growing capability, computer- 
aided paradigms have become so powerful that they can provide novel simulation- 
based assistance in many fields, for example, digital twins providing new services for 
predicting failures, increasing operational efficiency, or for service planning [76]. 

However, compared to computer-aided tools in engineering, computer-aided as- 
sistance by means of digital twins is a niche application. The manual setup of corre- 
sponding models is a tedious task requiring simulation experts. This limits the use of 
model-corresponding concepts since corresponding efforts and costs are major road- 
blockers for increased use [60]. Furthermore, the lack of rigorous concepts for quan- 
tifying errors often implies very conservative safety margins, so that the full poten- 
tial often cannot be exploited. Missing protection of intellectual property of models 
and the lack of standards (the functional mock-up interface [FMI] is only adopted 
slowly [10]) are hindering further. Thus today, digital twin-based approaches are only 
adopted in applications of high value, e. g., heavy-duty vehicles [44]. MOR [5, 4, 95] is 
a key technology to solve these challenges in the context of digital twins. By splitting 
computations in an offline and an online phase, computational effort is shifted to an 
offline phase allowing interactive simulation during the online phase. However, not 
only does this imply a speedup of calculations, but due to their reduced information 
set, reduced-order models (ROMs) protect intellectual property efficiently. While ge- 
ometries can be recovered from the meshes of three-dimensional simulations, this is 
not the case for ROMs, in particular since the output generally focuses on the quantity 
of interest, i. e., a temperature at a single location rather than a complete temperature 
field. This furthermore increases the usability, since only relevant information is ac- 
cessible. In addition, ROMs can be efficiently containerized using available standards 
such as FMI [10], thus increasing usability. In particular in view of the challenges laid 
down in Section 12.2, MOR is a key technology for digital twins. 

A variety of concepts and approaches have been introduced in the last decades 
mostly using projection-based approaches such as proper orthogonal decomposition 
(e. g., [111]), balanced truncation (e. g., [42]), the reduced basis method (e. g., [80]), 
or Krylov subspace methods (e. g., [7]). The key idea of most approaches is to reduce 
the space of considered functions by means of an appropriate low-dimensional basis. 
For (close-to-)linear models, MOR is state-of-the-art in computational engineering and 
science. For nonlinear models it is a highly active field of research (e. g., [8]). 
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In addition to classical MOR methods, machine learning offers an alternative ap- 
proach. Many successful applications, such as the efficient operation of wind parks 
[62], have been realized during the last years. Compared to model-based approaches, 
machine learning concepts require comparably little manual efforts to be set up. How- 
ever, being data-centric, machine learning is not applicable where only few data are 
available. This is often the case in industrial applications, where relevant data cannot 
be measured, cannot be shared (e. g., due to IP concerns), or is simply not available 
(e. g., failure data for small lot products). On the one hand machine learning could be 
used to speed up simulation models by means of learning the underlying simulation 
data (e. g., [43]), but a combined approach with the ROM as the foundation and ma- 
chine learning closing the accuracy gap seems to be a more promising approach [58]. 
However, such combined approaches have rarely been considered in the past and we 
believe that it has a strong future potential. 

Within the following sections we review the application of MOR in projects which 
tackled concrete aspects of the digital twin vision described above. However, before 
describing these projects we generally review the process of productizing algorithms 
since the overall goal of every industrial R&D activity is to improve or deliver new 
products or services. 


12.4 Model order reduction — from algorithms to 
products 


12.4.1 Introduction 


The process of making an algorithm suitable for use in commercial (CAE) software, 
also referred to as productizing, can be long and difficult to plan. Even if algorithms 
are known in the literature to be generally robust, the applicability to commercial soft- 
ware implementation is not always straightforward. In particular, it is challenging to 
foresee the user’s needs and desired application of a method so that the method’s as- 
sumptions do not lose validity. Moreover methods and software developers often face 
strict boundary conditions regarding implementation variants that are dictated by, 
e. g., the structure of the underlying physics engine or solver in which novel methods 
are implemented. Furthermore, while algorithms are usually designed by experts, the 
actual end-users are typically not experts in using those algorithms — they are experts 
in their own domain. Hence successful productizing requires not only that algorithms 
are robust with respect to applications, but also that their parameters can be (re)set 
in an automatic and dynamic way: automatic to reduce the need for users to set pa- 
rameters and dynamic because parameters may need to be adjusted not only at the 
start of but also during the simulation. In this way the numerical methods become 
transparent to the user while the freedom of the user to interact with the algorithm is 
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somehow restricted. A good balance between transparency and user freedom has to 
be found. The situation becomes even more complicated if a working algorithm is not 
available or if the problem at hand is not yet fully understood and analyzed. 

In this section we describe the various phases from algorithms to products. We as- 
sume that the problem to be solved is sufficiently well-defined (and constrained) and 
end-user requirements are known, and hence we focus on the process of solving the 
problem. As a concrete example, one could consider the typical MOR problem: given 
a dynamical system, find a reduced dynamical system that approximates the origi- 
nal system with a controllable trade-off between error and speed, and preservation of 
key properties like stability. We identify the following phases that will be discussed in 
more detail in the next subsections: 

— research: literature study and investigation of novel approaches; 

— prototyping: implementation of stand-alone or integrated software to allow feasi- 
bility studies; 

— productizing: implementation in, or as, a product; 

— customer feedback: closing the loop with new results and new requirements from 
end-users. 


These phases may overlap in practice and moreover the process might become iter- 
ative: After customer feedback, but also during productizing, often new insights are 
obtained which require further research and prototyping. 


12.4.2 Research 


During the research phase, traditionally two activities are dominant: literature study 
and design of novel approaches. Depending on the complexity and confidentiality of 
the problem, these activities are carried out by one or more researchers, e. g., a techni- 
cal leader, a (team of) researcher(s), and a MSc/PhD student, or even outsourced to an 
external party. For literature study, it is not only important to have the problem at hand 
well-defined, one must also know which literature to study. In some cases the right 
sources are naturally available because the researcher has experience on the topic. In 
other cases the topic may be less or even not covered in existing literature, or not in the 
context of the application at hand. Communication with colleagues (potentially in dif- 
ferent divisions) and external parties like universities is then required to at least find 
a starting point. In several cases such contacts, for instance made during conferences 
or European networks like EU-MORNET [30], may develop to long-lasting collabora- 
tions with rewards such as scientific and commercial breakthroughs and staffing op- 
portunities. The circuit simulation-related MOR work described in Section 12.10, for 
example, has been performed in collaboration with the TU Eindhoven, in the Euro- 
pean project ASIVA14 [21], while the drivetrain dynamics simulation tools described 
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in Section 12.8 have been developed in cooperation with the KU Leuven and the Uni- 
versity of Calabria within several years of research interactions and projects such as 
the Marie Curie H2020 project DEMETRA [57]. 

Often the problem is not sufficiently covered in the literature: The context or ap- 
plication may be different, the boundary conditions imposed by the main CAE solvers 
could be a limiting factor to the implementation of original algorithms, or the problem 
itself may simply be new. Even if the problem is well covered, one usually has to adapt 
and tune the proposed methods to the problem at hand. This stage, which may vary 
from simple changes of existing strategies to the design of novel approaches, typically 
involves prototyping, which we discuss in more detail in the next subsection. 


12.4.3 Prototyping 


When a set of methods is defined to achieve a specific target it is time to develop the 
first prototype code in order to test if the assumptions made during the research stage 
are valid and if the knowledge gained has application potential. In the prototyping 
phase, usually, one or more method developers and/or software engineers start to de- 
fine preliminary software architectures and begin the implementation of a prototype 
code. Common choices for development environments are MATLAB [66] and Python 
[79]. As a good practice, the developed mock-up code should be easy to extend, it 
should be tested in a similar environment as compared to the target solver in which the 
final implementation is foreseen, and it should be flexible enough to be tested in mul- 
tiple scenarios and maintain a satisfactory level of user-friendliness. In this way new 
extensions of the methods can be easily tested on multiple scenarios, the code can be 
shared with colleague researchers and consultants for usage in bilateral projects, and 
the risk of failure during the prototyping-to-product transition is reduced. Once the 
set of algorithms is mature enough, it is important to perform stress tests in the largest 
possible range of applications. Automatic testing is not mandatory but is surely an 
added value. 

Using the specific case of MOR-related algorithms, it can happen that a large 
amount of parameters must be set by the user and that these are of difficult physical 
and mathematical interpretation to nonexpert users. Moreover, automatic parameter 
tuning algorithms are rarely available in the literature for the specific application fore- 
seen for the implemented method. For this reason a big effort during the prototyping 
phase is generally spent in making the numerical methods robust and the automatic 
parameter setting transparent while still allowing advanced users to retain the de- 
sired level of control on the numerical method. During the prototyping phase of the 
method described in Section 12.8 the original number of parameters linked to the un- 
derlying MOR strategy was drastically reduced thanks to automatic parameter setting 
and the remaining parameters have been readapted to represent physical quantities 
that are easy to understand from a user point of view. Similarly, for the MOR approach 
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described in Section 12.10, most of the low-level parameters have been combined into 
macro-options that give the user (and developer) easy control over performance and 
accuracy. 

If this target is achieved, the prototype should be tested on real engineering cases 
during, e. g., bilateral services projects and/or funded research projects. This step is 
useful to confirm the potential of the method, find out unforeseen usages, and detect 
potential limitations. 

Often, at the end of the prototyping stage, a preliminary user interface is created 
to explore the usability of complex numerical solutions. 


12.4.4 Productizing 


Once the set of algorithms has reached a satisfactory level of robustness and usabil- 
ity the prototyping phase can be sided by the productizing phase. First the developed 
methods should be assessed for their market value, general applicability, and strategic 
importance. This stage is fundamental in order to assign a well-balanced amount of 
development resources. After this assessment the correct number of resources — gen- 
erally one or more developers and/or software engineers — is assigned the task of im- 
plementation into the target commercial CAE solver. The goal is to translate customer 
specifications, design requirements, and prototype code into a professional and con- 
sistent implementation. Especially during the implementation of novel methods, it is 
of paramount importance that researchers and developers communicate on a regular 
basis. In practice, the specific research knowledge and the application-oriented char- 
acter of many methods makes it hard to make consistent and complete code design 
specifications. In this case, developers may face the challenge of interpreting proto- 
type code and might implement nonintended behavior. It is advisable to initially allow 
researchers and developers to spend time together and even promote pair-coding ac- 
tivities. The more the algorithms are complex and have a dual theoretical-applied char- 
acter, the more this practice should be promoted. During this period and in parallel 
with the method implementation into commercial solvers, a team of developers might 
also start to implement a user-friendly user interface. The more the numerical method 
has been refined and made robust, the less the user interface creation process is chal- 
lenging. During the creation of the MOR method applied to drivetrains described in 
Section 12.8 a prototype user interface was also created in parallel with the research 
and method prototyping. This and the strong cooperation between the research and 
development units of Siemens allowed for a smooth transition of the prototype code 
and prototype user interface into a commercially available solution for MOR applied to 
drivetrain problems. One of the main challenges in the productizing of the methods in 
Section 12.10 was the choice on which parameters to make available to the user. This 
has been an iterative process itself, where researchers, developers, and application 
engineers were involved. 
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12.4.5 Customer feedback 


No matter how sound the underlying theory is and no matter how many tests have 
been done, the most useful feedback on the quality (performance, accuracy) of the 
product is end-user feedback. The difficulty, as mentioned before, is that the test cases 
used by development teams typically do not cover completely the real cases used by 
customers. Hence, there is always a risk involved with releasing improved or new func- 
tionalities. The key is again communication to manage expectations, not only inter- 
nally with sales and product engineering teams, but also with the customer (either 
directly or via customer-based application engineers): Roughly speaking, one of the 
first things to do when a customer request (bug report or enhancement request) is 
filed is to analyze whether there is a real bug in the theory and/or implementation, 
or whether the result is within accuracy tolerances but outside customer expecta- 
tions. Ideally this first analysis is done by application or test engineers, but depend- 
ing on the complexity, development teams may need to be involved as well. When 
the issue is identified as bug, apart from implementation errors, regularly one will 
have to go back to the underlying theory, for instance to adjust initially made as- 
sumptions or estimates, hence reiterating the phases described in the previous sub- 
sections. 

When the result is within accuracy tolerances but outside customer expectations, 
the situation can become more complicated. Not only one has to be sure that the result 
is indeed within tolerances, but one also has to explain this to the customer: Particu- 
lar care has to be taken here to avoid breaking long-standing trust relations. Further- 
more, it might also be an indication that certain settings and options in the software 
are not clear for users, which may require software and/or documentation to be im- 
proved. 

During the circuit simulation-related MOR work described in Section 12.10 all of 
the above-mentioned scenarios have happened. For example, a bug reporting a too 
large difference in signal delay was initially identified as a side effect of the way the 
delay was computed during postprocessing of simulation data. A deeper analysis, 
however, showed that while the actual delays were still within (user-settable) simu- 
lation tolerances, the used error estimations in the code were in fact too optimistic, 
and hence all phases above had to be reiterated in order to fix the issue. After the re- 
lease of the first version of the drivetrain simulation tools described in Section 12.8, a 
user signaled an extension request to improve the usability of the tool for large system- 
level models that include multiple drivetrains. The user was contacted and asked for 
feedback about the urgency of the required extension. It was then decided in agree- 
ment within the party to take the time to develop a proper interface for the requested 
extension and release it together with the official product release a few months af- 
ter. 
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12.4.6 Concluding remarks 


We conclude by repeating what was mentioned in the introduction: The phases de- 
scribed in this section are typically visited in an iterative way. Moreover, they may in 
fact be visited in any order, for instance when through the acquisition of software (or 
company) one starts with an actual product that has to be integrated in a larger envi- 
ronment. 


12.5 Use case - virtual sensors 


12.5.1 Vision 


The use of models to enhance or extend test-based engineering processes is one of the 
key application fields of model-based system testing [27]. Test data exploitation can 
be greatly enhanced by complementing sparse physical sensor measurements with 
model-based virtual sensor data [107, 20]. Control system efficiency can be increased 
by providing optimal control inputs using quantities which cannot be measured di- 
rectly and operating system performance can be tracked through monitoring internal 
system states. Traditionally such control inputs or internal states of devices are mea- 
sured during operation by hardware sensors [53]. However, due to cost restrictions or 
extreme physical conditions it is not possible to place hardware sensors at any desired 
position in any device. The goal of virtual sensors in all these applications is to provide 
online information about internal conditions or system performance based on simu- 
lation models instead of hardware sensors. These system models can be used offline 
to expand data sets or may be running parallel to operation, permanently synchro- 
nized with the current operation state, and report the desired internal states at the 
usual rate of the hardware sensors. From a business perspective such virtual sensor 
software modules may not only add value to the engineering process but can enable 
new simulation-based products such as advanced condition monitoring for improved 
availability or reduced downtimes. Furthermore, when virtual sensor algorithms and 
existing controllers are integrated into one software architecture, novel model-based 
controllers can be realized. 

However, the systematic application of embedded simulation models for ex- 
tended data analysis or parallel to operation is still a young field of activity. On the 
other hand, driven by the need to reduce development cycle times, simulation has be- 
come a frequently used tool during the development of products [17]. To draw reliable 
conclusions during the development process detailed three-dimensional simulation 
models are needed and the evaluation of these simulation models typically involves 
significant simulation times. This makes their reusage inside virtual sensor software 
and related state estimation a challenge. 
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In fact one of the central requirements for simulation models inside virtual sen- 
sors is the capability for fast estimation or even real-time capability when the results 
should be updated within the usual update frequency of hardware sensors. For this 
reason, MOR [8, 5] is applied to, e. g., detailed three-dimensional simulation models 
developed for design engineering purposes. This ensures reusage of the already avail- 
able information and it allows to obtain fast or even real-time capable surrogate mod- 
els which nevertheless operate within an acceptable accuracy. 


12.5.2 Technological challenges 


In this section the required steps for a virtual temperature sensor are described. 

For a virtual temperature sensor the starting point is the thermal energy equa- 
tion which reads for heat conduction with Fourier’s law q = —KVT [67, 55, 105] fora 
computational domain Q as 


0,(C,T) + V-(-KVT) =h in Q, 
q:n=hy onTy, (12.1) 
q:n=a(T -T mb) onTp. 


Here, T is the temperature field, T mp is the ambient temperature, Cp is the specific 
heat capacity, x is the heat conductivity, and a is the convection coefficient [61, 56]. 
In a typical industrial setup, Dirichlet boundary conditions are not used. Instead, the 
thermal losses are captured by the volume heat load h or the heat fluxes hy at the 
boundary. The most important boundary condition is the Robin boundary condition, 
which is also known as Newton’s law of cooling [56]. This boundary condition models 
the thermal communication with the environment. Especially when a thermal model 
contains only solid bodies which are surrounded by a coolant, the convective heat 
transfer coming from the coolant flow can be modeled by a given distribution of con- 
vection coefficients. For example, this applies to thermal models of electric motors 
which contain the solid parts of the stator, rotor, and housing, but not the flow do- 
main of the cooling air flow. 

To start the MOR procedure, the thermal energy equation has to be written as a 
state-space system in the form 


d 
E—x=Ax+Bu, 12.2 
ET. + (12.2) 


y=Cx. 


1 A full conjugate heat transfer model would lead to a dramatic increase in complexity and compu- 
tational time, since the turbulent and thermal air flow in the rotor-stator gap and around the stator 
cooling fins needs to be resolved [56]. 
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Here x € R” is the system state, u € IR” is the input which drives the system, and 
y € R is the measurable respectively observable system output. Furthermore, the 
system matrices are of dimensions E,A € R®”, B € R™™, and C € RP”. 

To obtain the thermal energy equation (12.1) as system (12.2), the following steps 
need to be performed. 

— The heat load h and heat flux hy are assumed to consist additively of contributions 
which only vary in time, i. e., h = hy(t)+---+h,(t) and hş = hy y(t) feet hf x(t). This 
assumption is fulfilled for a typical thermal simulation modelin the industry since 
the usual procedure in commercial three-dimensional simulation software is (a) to 
mark the relevant model components on which the heat loads and the heat fluxes 
are applied and (b) to specify the total thermal losses which are produced by these 
model components. In a subsequent step the commercial software distributes the 
total thermal losses spatially homogeneous over the marked model component 
[2]. This leads to m = 1+ k inputs. 

— A finite element method or finite volume discretization approach in space brings 
the thermal energy equation almost into the desired state-space formulation. 
Some minor changes are necessary since during the finite element method or fi- 
nite volume assembly procedure a constant vector by occurs at the right-hand side 
due to the Robin boundary condition [59]. This part is added to the input terms 
by extending the input matrix B as B = (B, bọ) and the input vector u as u = (u, 1). 
This leads in total tom = l + k + 1 inputs. 

— Additionally the output matrix C has to build up according to the desired location 
of the virtual sensors. This is done by marking for each virtual sensor its relevant 
nodes or elements in the computational mesh. This determines for each virtual 
sensor its corresponding row in the output matrix C. 


The major technological challenge in this process is to access the assembled system 
matrices from commercial CAE software. For Simcenter Thermal Flow [2, 99] this was 
solved with a special subroutine and for NX Nastran [2, 71] this was solved with DMAP 
[2, 25]. However, there are commercial CAE software packages which do not provide 
any customization possibility to access the system matrices or some explicit solvers 
even do not assemble global system matrices. 

Once the state-space system corresponding to the thermal simulation model is 
obtained, any MOR method can be applied which works on state-space systems of 
type (12.2) [8, 5]. For thermal simulation models the matrices are huge in size but 
sparse [59]. In our experience, typical industrial small-sized thermal models contain 
up to 10° degrees of freedom and typical medium-sized thermal models contain up 
to 10° degrees of freedom. For this reason the Krylov subspace MOR methods are a 
good choice since Krylov subspaces have a long history in connection with linear it- 
erative solvers for especially huge and sparse linear equation systems [39]. A detailed 
review of Krylov subspace MOR methods can be found, e. g., in [5, 8, 7, 72]. Instead 
of giving yet another introduction into Krylov subspace MOR methods we concentrate 
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on what is necessary to realize virtual sensors with these methods in an industrial 
environment. 

However, classical Krylov subspaces MOR methods such as [72] are feasible only 
for linear and time-invariant systems (12.2). For products with temperature-dependent 
material properties, the heat equation (12.1) becomes nonlinear due to x = x(T). In this 
case nonlinear algorithms (e. g., [8, 114, 69]) need to be applied. 


12.5.3 Project description 


The first goal was to establish a user-friendly work flow for generating ROMs from ex- 
isting three-dimensional thermal simulation models in an industrial environment. For 
this goal the determining factors are that (a) the simulation models are constructed in 
a commercial CAE software and (b) the simulation engineers have profound knowl- 
edge in their physical domain and the used CAE software but in general they are not 
experts in MOR nor they are programmers; see Section 12.4 for more details. More pre- 
cisely, since all commercial CAE software packages are used through graphical user 
interfaces, simulation engineers are generally not used to run algorithms in command 
line tools or software development environments.’ This starting position requires (a) 
to interact with the commercial CAE software and (b) to hide the details of the MOR al- 
gorithms from the user (Section 12.2). For this reason a MOR plug-in for Simcenter [26], 
the flagship product of Siemens in the CAE market, was developed. This MOR plug-in 
adds a ribbon to the Simcenter Graphical User Interface (GUI) which guides the user 
with buttons and following pop-up windows through the process of generating, apply- 
ing, and exporting ROMs (Figure 12.3). This MOR plug-in was developed as Siemens 
internal engineering tool and is in productive usage within different projects and de- 
partments. To ensure that the resulting GUI matches the user expectations, several 
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Figure 12.3: Model order reduction plug-in. 


2 The main task of simulation engineers is to support or enable the product development process 
based on simulative information. To accomplish this, simulation models with the relevant physical 
information are built from CAD models. From the obtained simulation results conclusions are then 
drawn, e. g., about the product design or the reliability, and this information is fed back in the devel- 
opment process. This means that simulation engineers are focusing on product development and not 
on algorithm development. 
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in-house simulation engineers were included in the process of designing the GUI and 
the work flow of the plug-in (Section 12.4.5). With this plug-in the step to easily gener- 
ate ROMs from existing three-dimensional thermal simulation models was solved. 

For realizing virtual temperature sensors, the next step is to wrap the obtained 
ROM inside a virtual sensor software module which is runnable on the target hardware 
and software architecture. For the communication with the surrounding software ar- 
chitecture, the virtual sensor software module must receive the current operating con- 
ditions, transform these conditions into the required input for the ROM, call the ROM, 
transform the ROM results into the required format, and feed the properly formatted 
ROM results back into the surrounding software architecture. Furthermore, one com- 
munication cycle of that kind must be done within an expected frequency. 

A crucial point is the available information during operation. Typically the avail- 
able information is not identical with the required input for the ROM. For example, 
for electric motors the current is known during operation and can be fed into the vir- 
tual sensor software module. However, the ROM obtained from the thermal simulation 
model requires heat loads as inputs. Thus it must be part of the offline phase, i.e., the 
creation phase of an ROM, to provide the required information for mapping the avail- 
able inputs (e. g., current) to the required ones (e. g., heat loads). This task involves 
detailed product-specific knowledge and is a central key for a vital and accurate vir- 
tual sensor software module. In our projects this task was solved with detailed look-up 
tables which were provided by the respective engineering departments. 

Another important ingredient ofa virtual sensor software module is to ensure that 
the ROM is permanently synchronized with the actual operation condition of the prod- 
uct. This requires that the virtual sensor software module receives and adequately pro- 
cesses the relevant information about the current operation state to keep its internal 
ROM synchronized. In our projects we solved this task with online filtering algorithms 
[101, 100, 46, 52], such as Kalman filters, where the filtering was done based on the 
available temperature hardware sensors and the corresponding temperatures coming 
from the ROM for these locations. 

The last step in the development procedure is to run system tests to improve the so- 
lutions based on this feedback. Some of our projects have currently reached this stage, 
whereas in our in-house hardware lab virtual temperature sensor software modules 
are already running and tested. 


12.5.4 Results and summary 


The main challenge of our projects was that virtual temperature sensors based on 
ROMs were realized for the first time for the considered products. This means that not 
only the software modules had to be developed but we also had to establish a work 
flow of how to realize these virtual sensor software modules. While customized one- 
time solutions are sufficient for research projects and first prototypes, they are not a 


394 —— D.Hartmann etal. 


proper solution for new services or products. New products or services require a sus- 
tainable work flow which is integrated into the existing development ecosystem of the 
involved engineers (Section 12.4). The approach we put into practice started from ex- 
isting three-dimensional thermal simulation models. These models were compressed 
with MOR and the resulting ROMs were small and fast enough to be executed within 
the usual hardware sensor update frequency, either in an embedded environment or in 
a cloud environment which is connected to the product. In order to integrate this task 
in the existing development ecosystem of simulation engineers, we developed a plug- 
in for Simcenter, which is the standard CAE software within Siemens for simulation- 
based engineering steps. 

The following task of integrating the ROM into the target hardware and software 
system was still realized as customized and manual solution for each product. A po- 
tential future integration of this step into the existing development ecosystem of au- 
tomation engineers are new state-observer blocks within the Totally Integrated Au- 
tomation portal, which is the engineering platform from Siemens for all kinds of au- 
tomation tasks [104]. During our projects prototypical blocks for such state-observers 
based on ROMs were developed but a fully integrated solution is still pending. Nev- 
ertheless, exactly the integration into existing automation engineering software tools 
is the second important step in realizing virtual sensors in a standard way. Overall, 
in a typical industrial development ecosystem, the simulation engineers create the 
ROMs for virtual sensors and the automation engineers integrate the virtual sensors 
into the software architecture of the products. Thus, to establish virtual sensors based 
on ROMs there must be a fully integrated solution for both, the simulation and au- 
tomation engineering ecosystems (Section 12.2). 


12.6 Use case — predictive maintenance 


Data-driven operation support has been a topic for about 10 years. The efficiency of 
methods such as condition-based monitoring or sensor-based fault detection depends 
on the amount and the placement of sensors. 


12.6.1 Motivation for model-based predictive maintenance 


Very new is the demand of simulative operation support [29]. It allows monitoring ev- 
ery position and physical size of a system at any time point. Due to this knowledge, 
the system state may be predicted at any time. A simulation-based software program 
runs in parallel to the operation and is synchronized by sensor values at every time 
point. In many reports this is called the digital twin. The benefits are summarized in 
Figure 12.4. Among the most important benefits are inspection and service planning, 
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Figure 12.5: Advantages for offering a digital twin additional to the hardware. 


lifetime prediction, advanced fault detection, and control and optimization during op- 
eration. Selling not only the hardware of the system but also additional services can 
be a huge advantage in countries with high salaries. There are existing first clients 
of Siemens who demand this kind of operation support. In a first view some services 
such as giving an availability guarantee may sound risky for a company. On the other 
hand this is a unique selling point and selling the risk may bring good profit; see any 
insurance company. A more accurate analysis leads to the conclusion that all partic- 
ipants may benefit from an availability guarantee (Figure 12.5). Giving an availability 
guarantee for products cannot mean that there are no downtimes due to faults or in- 
spections. Instead, the downtimes, especially the unexpected downtimes, should be 
reduced. The main task is to detect faults at a very early stage and predict their degra- 
dation. Thus, immediate downtimes are transferred to predictive downtimes. The base 
for giving an availability guarantee is the early detection of faults. If a fault is detected, 
then its degree of degradation is predicted. Depending on this prediction an inspec- 
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tion may be scheduled and/or the performance of the system is reduced in order to 
achieve the inspection time. Often, the plant is located in very isolated regions. Thus, 
the execution of maintenance and spare part supply must be planned very carefully. 
The early knowledge of the cause of failure is of tremendous interest. 


12.6.2 Oscillatory mechanical systems 


We consider a solid body Q c R? with boundary 00 = Tp UT y, composed of a material 
with Young’s modulus E > 0 and Poisson ratio —-1 < v < 0.5. The body is subject to 
volume forces f : Q — R? and surface forces g : dQ > R’. Displacements d : Q — R? 
from some appropriate function space €(Q, R°) are determined by the equations of 
linear elasticity (see, e. g., [49]): 


—div(Ae(d))=f inQ, 
(Ae(d))-n=g  onTy, (12.3) 
d=0 on Tp, 


where the strain e(d) is given by the symmetrized gradient of displacements, 
e(d) = 5 (vd + Vd") € R?, (12.4) 


and the stress Ae(d) is given by 


Ae(d) = 2ue(d) + A trace(e(d))I (12.5) 
= 2ue(d) + Adiv(a)I. 
Here, A = amin and p = in zy are the Lame constants and I is the identity matrix. 


Equation (12.3) is the strong formulation for linear static elasticity. A Galerkin dis- 
cretization of the weak formulation (typically by finite elements) yields a linear system 


Kd=f, KeR™", af eR", (12.6) 


where n is the dimension of the ansatz space, K is the stiffness matrix, and, by abuse 
of notation, d is the vector of displacements and f the vector of acting forces. 

In the dynamic case, i.e., when d and f are time-dependent, equation (12.6) is 
extended to [113] 


Md + Di(t)d + K(t)d = f(t), (12.7) 


where M € R™" is the mass matrix and D € R" is the damping matrix. Note that 
both D and K may be time-dependent. An important special case of this is the rotor 
dynamic equation 


Md + (D(w) + wG)d + K(w)d = f (t,w), (12.8) 


where w denotes the angular velocity of the rotor and G is the so-called gyroscopic 
matrix [36]. 
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12.6.3 Model order reduction 


In many real-world applications, the number n of degrees of freedom of the discretized 
system (12.7) or (12.8) is large and its numerical integration is not possible in real-time. 
MOR strategies introduce a reduced state q € R” with r « n, via d = Yq, Y € R”’. 

One way to obtain the reduction matrix ¥ for system (12.7) is to use modal reduc- 
tion. Setting up the eigenvalue problem of equation (12.7) 


w’MO = KO (12.9) 
and taking the first r eigenvectors, the matrix Y may be defined by 
W = {0,,...,0,}. (12.10) 


A preferable technique may be the Krylov subspace methods [7, 93]. The subspace is 
defined by 


Y = {K,f,K,|MK,}f,...,(K,1M)" Kf}. (12.11) 


The Krylov basis may be computed by the Arnoldi algorithm, which delivers an or- 
thonormal basis of the subset. 
Inserting this into (12.7) and multiplying by Y”, one obtains the reduced equation 


M@q + Dit)q+ Oq = YF, (12.12) 
where 
M=¥'MY, (12.13) 
D=W'DY, (12.14) 
K=¥'KW ce R” (12.15) 


are the reduced matrices. In the case of rotor dynamics, D and K depend on w. In 
the ramp-up phase of an electric engine, where the rotation frequency increases, the 
reduction operations (12.14) and (12.15) have to be performed in each time step. Inter- 
polation schemes may reduce the computational effort. In the constant phase of the 
engine, also the reduced matrices remain constant. 

As filtering methods are generally applied to first-order equations, we define as 


usual 
=(4)) #=( je ) 
x= 5 > u(t)= . ; 12.16 
& O=( 4 (12.16) 
and, assuming that M is invertible, 
ae =( : i ) B-( y ) (12.17) 
“Ua kO mw Dt)” Vw? 


so that we obtain the equivalent first-order system in state-space form 


x = A(t)x + Bu(t). (12.18) 
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Figure 12.6: Electric engine/generator configuration of about 20 MW. 


12.6.4 Fault detection in terms of unbalance of a rotor 


Following [13], [64], and [63] we want to identify an unbalance of a rotor during op- 
eration. The test configuration is an electric engine which drives directly a generator 
(Figure 12.6). The two rotors are connected by a clutch. Starting point for the analysis is 
the rotor dynamic model which was used in the design process of this particular driv- 
etrain (Figure 12.7). According to the strategies described in Section 12.6.3, we reduced 
the model in order to obtain real-time capability. In order to obtain realistic frequen- 
cies for the model, also some nonlinearities in terms of the fluid bearings have to be 
considered. Therefore, a nested procedure was applied [86], which keeps the nonlin- 
ear parts at the bearings and reduces the linear parts in term of the motor and the 
generator. Both rotors from the considered drivetrain are equipped with four discrete 
planes meant for balancing the rotor. The validation of the digital twin was done by 
physically attaching a small test weight to one of the balancing planes. Four sensors 
located at the bearings (red bars in Figure 12.7) provided the measurement data which 
are compared to the simulation results. 

The comparison or identification was performed by an augmented nonlinear 
Kalman filter procedure. The unbalance itself enters the model in terms of an external 
force (see equation (12.7)), where location, orientation, and magnitude are identi- 
fied by the filtering algorithm. The result of the identification method is presented 
in Figure 12.8. The blue peak in Figure 12.8 presents the location and the amount of 
unbalance. 
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Figure 12.8: Unbalance detection of the rotor during operation. 


12.6.5 Summary 


By combining MOR techniques and nonlinear identification methods, a digital twin 
for detecting and localizing faults (in terms of unbalances) has been developed for 
rotating systems. Further efforts are made in order to predict the increase of vibra- 
tion during operation. The time a critical vibration is achieved defines the moment 
for scheduling an inspection. Knowing this time at an early stage, an inspection and 
spare part supply can be prepared. 


12.7 Use case — operation control 


12.7.1 Engineering controlled systems 


The product race has become an innovation race, reconciling challenges of branding, 
performance, time-to-market, and competitive pricing while complying with ecolog- 
ical, safety, and legislation constraints. The answer lies in “smart” products of high 
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complexity, relying on heterogeneous technologies and involving active components. 
The corresponding design and engineering process hence must take the integration of 
control functions in the product explicitly into account. This adds an important addi- 
tional complexity to the design engineering process where the interaction between the 
control and the system requires these should be optimized concurrently. The current 
industrial practice however still treats passive system design and controller design as 
different and separate design loops with their own models and their own validation 
and verification strategies. Suboptimal designs and unexpected integration problems 
are the result. Not reusing the wealth of engineering models available from earlier 
detailed system design stages furthermore leads to inconsistency problems and inef- 
fective engineering processes. Closing this gap offers a significant potential for opti- 
mized designs, better product performance, and fewer and shorter design iteration 
cycles [1, 106]. 


12.7.2 Technological challenges 


Two classes of challenges can be distinguished in relation to the integration of con- 
trol functions in the product. The first one targets the optimization of the controller 
architectures, strategies, and settings for a controlled product, hereby using a sys- 
tem or “plant” model in a virtual controller optimization process. The second chal- 
lenge targets the design of an optimal control solution by including a model of the 
controlled system into the controller itself, for example in a model predictive control 
(MPC) approach [82]. For both cases, the used system models are typically developed 
dedicated for the control application taking into account feasible complexities and 
system simplifications. One objective to do so is to allow fast virtual testing and opti- 
mization cycles and to enable hardware validations in physical control environments. 
The detailed (for example multiphysics) design engineering models from the system 
design departments are typically not reused. The main reason for such a suboptimal 
approach is that either detailed system models from the system engineering design 
departments are not available in the control department or are of too high complexity 
to be used together with control simulation. 

Focusing on the first challenge of optimizing the controller design, one can dis- 
tinguish three phases: 

Phase 1: The combination of the multiphysics simulation model with that of the 
controller enables the design of the control logic and the performance engineering 
of the intelligent system. This is referred to as “model-in-the-loop” (MIL). The simula- 
tion is offline, i. e., there is no requirement for real-time performance of the simulation. 
Basically, two interconnecting objectives can be distinguished: One is to perform sys- 
tems engineering based on the multiphysics “plant” model, including the representa- 
tion of (often simplified or idealized) control in the multiphysics model; the other is 
to perform control engineering, using a model of the system to be controlled (“plant 
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model”). The first objective, for example, serves the purpose of configuration design 
(how many actuators and sensors, where to place them, etc.) or concept evaluation 
studies or the optimization of the mechanical system design taking into account the 
presence of control and certain control laws. The second objective is oriented towards 
the development of the optimal control logic and the development and verification 
of control hardware, control libraries, and embedded software up to the validation 
and calibration of the control system on the electronic control unit (ECU) (Figure 12.9) 
[106]. 
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Figure 12.9: Associative plant models for control engineering. 


To couple the models, different approaches exist. One may embed state equations 
with a description of the plant system (e. g., multibody simulation models or one- 
dimensional ordinary differential equation-based system simulation models) into 
these of the control (or vice versa) to enable the use of one solver, or adopt a true 
co-simulation approach where each system part runs its own solver [106, 40]. 
Alternatively, or in combination with the above approaches, a reduction of the 
plant model (e. g., a finite element or complex, even nonlinear multibody simulation 
model) into a description compatible with the controller model (e. g., state-space for- 
mulation) can be used. The model reduction step mostly achieves its goals at the ex- 
pense of the full observability and/or controllability of the physical phenomena, lead- 
ing to a macroscopic “equivalence” but loosing direct insight in the microscopic ob- 
servation domain. The challenge is to develop model compression methodologies that 
allow maintaining a relation with the physical meaning of model parameters. Such 
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co-simulation and model reduction approaches are used both for MIL applications 
for systems engineering and for control logic engineering. 

Phase 2: The next step is the development and optimization of the “embedded” 
control software. This needs also to be done in the context of the functioning of the 
multiphysics system to be controlled. This is referred to as “software-in-the-loop.” 
While some of this can be done in offline simulation (provided software libraries of the 
controller are available), the final optimization needs to take into account the work- 
ing of the software in real-time, requiring real-time capable multiphysics simulation 
models. 

Phase 3: The final testing and calibration of the controller software and hardware 
requires the controller to be connected to a multiphysics simulation model of the com- 
ponents, subsystems or system, in a dedicated computing environment that is referred 
to as “hardware-in-the-loop” [6]; of course, this requires real-time capable simulation 
models. 

The use of MOR is hence a key factor for enabling a true model-based engineering 
approach where consistent engineering models can be used throughout the various 
design phases. The applied MOR methods may depend on the reduction purpose and 
the nature of the master models. For example, in mechatronics systems, these can be 
finite element, multibody, or multiphysics models which can be linear or weakly or 
strongly nonlinear. Two application cases will be briefly discussed. 


12.7.3 Application to the design of an active sound quality control 
system 


Active noise reduction (and sound shaping) is a widely studied research topic with 
many potential industrial applications. Structural-acoustic solutions using smart ma- 
terials as sensor and/or actuators are explored, enabling intelligent structures. Such 
solutions are however typically developed as add-on systems which prevents optimiz- 
ing their potential impact as part of the overall system. A model-based mechatronic 
engineering approach was developed to enable an integrated solution [22, 23]. It was 
applied to the active sound control of vehicle engine noise to the car interior. The 
challenge consisted of relating the large three-dimensional, frequency-domain (finite 
element- and boundary element-based) vibro-acoustic, and structural models for the 
vehicle structure and structural components, interior vehicle cavities, and exterior 
propagation field, with models of smart material sensors and actuators and a time- 
domain control model. A simplified vehicle structure was developed to allow experi- 
mental validation. It consisted of a concrete car body, an engine compartment with an 
artificial source, and a flexible firewall panel on which the structural-acoustic control 
was to be applied with piezo-elements. The rigid walls of the concrete car structure can 
be easily modeled and also treated with a dedicated damping surface (Figure 12.10). 
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MOR was a key element in the modeling approach, allowing to incorporate the re- 

duced model as a plant model in the controller simulation. The component mode syn- 

thesis (CMS) approach was used. Very large reduction factors were used, reducing the 

large structural/vibro-acoustic finite element model (25,000 acoustic degrees of free- 

dom but which can overall easily reach hundreds of thousands of degrees of freedom 

when multiple flexible panels are included) to a time-domain state-space model of 

realistic size (200 degrees of freedom). The sensors and actuators were represented 

by one-dimensional models for their functional performance, while their added mass 

and stiffness are accounted for in the three-dimensional finite element models. The 

acoustic propagation was related to the structural outputs by means of an “acoustic 

transfer vector” approach. The modeling approach included the following steps using 

multiple software tools: 

- generate structural mesh and apply material properties (finite element preproces- 
sor); 

- add actuator and sensor mechanical models (finite element preprocessor); 

—- run a modal analysis (finite element analysis); 

- build the acoustic finite element model and perform modal analysis (finite ele- 
ment analysis); 

- import the structural model and couple it with the acoustic one (finite element 
analysis); 

- calculate actuator and sensor electromechanical coupling (extended finite ele- 
ment analysis); 

— reduce and convert the finite element model into a state-space model (MATLAB); 

- implement and optimize the controller with the coupled state-space model (MAT- 
LAB/Simulink). 


The coupling between acoustic and structural models is shown in Figure 12.11. 

After performing a coupled modal analysis, the desired degrees of freedom are 
taken to derive the state-space model. In this case, the state-space model features two 
inputs (one actuator on the firewall and a sound source in the engine compartment) 
and four outputs (three pressures in the passenger compartment and one velocity on 
the firewall). The state-space model derived from this coupled approach allows the 
implementation of any controller involving the predefined degrees of freedom, and 
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Figure 12.11: Coupled structural acoustic models to be reduced to a modal basis. 
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if the finite element approach involves the systematic representation of the sensors 
and actuators, the resultant state-space model is, in fact, a representation of the fully 
coupled electro-vibro-acoustic system, with any possible input-output relationships 
allowed by the chosen degrees of freedom. 

Using this model, an optimization procedure is performed. The cost function takes 
into account the sound pressure level at the drivers head, the actuator input energy, 
and the weight of the solution. The firewall thickness and the velocity feedback con- 
troller gain are the variables. The position of the collocated sensor/actuator pair (SAP) 
can be considered fixed or included in the optimization loop. Figures 12.12 shows the 
cost function for each thickness as a function of the feedback gain, on the best SAP 
position for each case. 
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The best SAP position and optimal feedback gain depend on the thickness, which 
indicates that the global optimum can only be achieved in such a concurrent design 
between the active and passive system characteristics, proving the effectiveness of an 
integrated mechatronics simulation approach. The same model can be used to evalu- 
ate different controller strategies such as combined feedforward/feedback, filtered-X 
LMS, and NEX-LMS, and for different performance targets (noise level optimization 
and/or sound quality control). A more extensive discussion of the various modeling 
aspects and the detailed optimization procedures can be found in [22, 23]. 


12.7.4 Application to control development using neural 
network-based model reduction 


The development of the control of mechatronic systems becomes more complex when 
multiple actuators and sensors are interfacing with a highly dynamic multiphysical 
system, often not having the sensors available to develop an optimal control. In the 
automotive industry, controls have mostly been developed using rule-based meth- 
ods, first by directly writing code, later using a model-based approach. In the au- 
tomotive industry, the complex balancing of multiple performances of the combus- 
tion engine such as emissions, fuel economy, and acceleration performance have in- 
creased the complexity of the engine actuators. To develop the controls for such com- 
plex mechatronic systems, new methods are required. Optimal control such as MPC in 
combination with methods to predict virtual controllable quantities using state esti- 
mation technologies in combination with Kalman filtering are examples of such new 
technologies that start to find their entry in the automotive world. MPC and Kalman 
filtering-based state estimation require, however, models that run fast while keeping 
acertain level of accuracy. Often, ad hoc simplified models are (re)developed, neglect- 
ing the availability of detailed engineering models. Reusing such models would not 
only save modeling time but would also allow better consistency of the various design 
engineering models over the different attributes and product versions and variants. 
The simulation models for designing mechatronic systems are often created based 
on a combination of detailed three-dimensional models and test data and have a high 
level of accuracy but have a too slow calculation time to be used in MPC or state es- 
timation methodologies in real-time on an ECU. To be able to convert such system 
models into the context of optimal control in combination with virtual sensing, neu- 
ral networks can be an ideal methodology to develop a control model directly from 
the detailed plant model to be controlled [50, 65]. Figure 12.13 explains the different 
steps in the process, showing the reduction of the detailed engine model to a neu- 
ral network-based model for the virtual sensor as well as the optimal controller. The 
neural network ROM allows real-time execution of the model for both the sensing and 
the control action. Initial results using the controls model in closed loop with the de- 
tailed plant model for tracking purposes indicate that the performance, for scenarios 


406 —  D.Hartmann etal. 


z Pa 
ed va 


= N 
i DUNUDENA 


ZER, 


E 


, 


Exogenous 
inputs 


MPC 
Controller 


Controls 


Measure 


Figure 12.13: Advanced combustion engine (upper left), one-dimensional system model (upper 
right), and reduction by neural networks in the real-time virtual sensor and MPC controller models. 


for which the neural network is not trained, remains within good accuracy as long as 
the important states of the model are kept observable. Further results in this field will 
bring more clarity in how broadly this technology can be used for engine controls or 
other advanced vehicle controllers. 

By applying multiple load cycles covering the full operating space of the system, 
the neural networks can be trained to represent the relevant system behavior even in 
the case of strongly nonlinear system characteristics. 


12.7.5 Summary 


Model-based approaches find increasingly their way into the design of (optimal) con- 
trol systems. In the majority of cases, however, the applied system models (“plant 
models”) are ad hoc developed low-complexity models that are not correlated to the 
design engineering models developed in the mechanical design stages. This not only 
leads to inefficient processes redoing efforts that could be recovered, but also gives 
rise to major issues related to consistency and traceability when design improvements, 
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versions, or variants are to be processed. MOR can offer an answer for both the control 
design and control implementation and opens up new opportunities for concurrent 
design of the mechanical and the control system. Major challenges are still presented 
by the very large reductions factors to allow fast control optimization or even real-time 
usage inside state estimators or MPC controllers. The use of a neural network-based 
approach to reduce complex nonlinear models subject to an envelope of operating 
conditions offers significant potential that is to be further investigated. 


12.8 Use case — drivetrain analysis 


12.8.1 MOR for contact mechanics problems 


The simulation of dynamical systems involving contacts between elastic bodies [112] is 
a challenging and active research field on its own. In particular high-frequency phe- 
nomena, numerical stiffness, high degree of nonlinearity, high dimensionality, and 
multidisciplinary nature (mechanics, acoustics, fluid dynamics, tribology) are among 
the major challenges that researchers and software developers must address in or- 
der to efficiently solve these types of problems [11]. Despite its seemingly niche de- 
scription, there is a wide range of applications in which these problems are found 
and need to be solved. In particular, the simulation of geared transmissions or drive- 
trains is practically ubiquitous if one has to deal with simulations of electromechan- 
ical machines. Drivetrains contain a multitude of components, including bearings, 
gears, clutches, and spline connections that are known to behave nonlinearly and 
contain multiple contacts between flexible objects. While several MOR methods have 
been applied largely and successfully [32] in the field of flexible multibody simulations 
[97] in both academic and industrial settings with a large growing body of literature, 
MOR methods dedicated to the field of contact mechanics have been only recently ex- 
plored [12, 103]. Moreover the developed methods often target high-dynamic contact 
mechanics simulations with fully flexible bodies and dynamic interactions with flexi- 
ble eigenmodes of the structure [12]. These problems would indeed remain practically 
intractable without the usage of MOR and/or computer clusters. On the other hand, a 
large set of system-level related problems (such as large drivetrains or more complex 
machines containing several drivetrains) might not need the level of fidelity and the 
still relatively large computational times necessary to solve a fully nonlinear dynamic 
problem. They might instead still benefit from MOR solutions that speed up simulation 
times and decrease memory usage and disk storage, while still retaining the required 
level of fidelity on both system and component levels (Section 12.2). 

The method discussed in this chapter exploits both an advanced MOR strategy 
and physics-based considerations coming from the targeted application domain of 
drivetrain simulation and combines them. The result is a numerical strategy that is 
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efficient and computes accurately most of the drivetrain dynamics-related scenarios 
that are industrially relevant. The focus is on three-dimensional system-level simu- 
lations including lightweight and internal gears and noise-vibration and harshness 
(NVH) problems in a multibody simulation environment. 


12.8.2 Technological challenges 


While the application challenge is relatively straightforward to summarize — efficiently 
and accurately solve three-dimensional multibody problems involving multiple gear con- 
tacts for system-level and NVH purposes — the technical challenges connected to it are 
multiple and have their root in the mathematical description of the equations of mo- 
tion of a multibody system. The following set of equations is an index 3 differential 
algebraic equation describing the dynamic motion of a flexible multibody problem: 


M(x)x + Kx + G'A = faxt + fy (12.19) 


where M(x) is the nonlinear mass matrix, K is the linear stiffness matrix, x is the vector 
of generalized coordinates, G is the Jacobian of the constraints, A is the vector of La- 
grange multipliers, and f,,, and f, are the vectors of external forces and the quadratic 
velocity terms. Without entering in more details (which can be found in, e. g., [11]) we 
mention that despite being a fully nonlinear problem, the equations describing the de- 
formation of the flexible bodies present in the system can be reduced by using linear 
MOR methods for second-order systems in the following form: 


Mü + Diu+Ku=f, (12.20) 


where M, D, K are the linear mass, damping, and stiffness matrix of the underlying 
finite element model, u is the vector of linear nodal deformations, and f is the vec- 
tor of nodal forces. This system can be reduced thanks to Petrov-Galerkin projection 
methods: 


W'MVii+W'DViu+ W'KVqu=W'f, (12.21) 


where W and V are the left and right subspaces used to obtained the reduced system. 
Galerkin methods can be used in flexible multibody problems within the assumption 
of large gross motion but small deformations within each of the body frames. This 
assumption is amply satisfied in problems involving gear contacts. 

While flexible bodies that are not involved in contact interactions can be reduced 
with very efficient techniques, such as balanced truncation [83, 42], Krylov [32], CMS 
[19], etc., flexible bodies that include contact interactions suffer from the so-called in- 
terface problem [102]. In practice, a very large and inefficient reduction space needs 
to be used for an accurate reduction. The size of the reduction space becomes propor- 
tional to the amount of degrees of freedom potentially involved in the contact interac- 
tions. This causes problems such as high memory usage, large precomputation time, 
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large storage requirements, and increased numerical stiffness. Finally, the contact de- 

tection phase is also computationally very costly and scales with the (large) number 

of degrees of freedom that can be involved during contact. 

In recent years several methods have been presented to maintain a high level of 
accuracy — similar to nonlinear finite element full-order dynamic computations — but 
drastically limit the impact of the above-mentioned issues. In particular, the following 
works [11, 12] obtain very good results in terms of speedup and memory usage while 
losing only a fraction of the accuracy obtained with nonlinear finite element problems. 
The field of hyperreduction [18, 31] is also exploited to tackle the contact detection 
problem with very promising results. However, this methodology is relatively complex 
to include in proprietary multipurpose multibody solvers and, moreover, the simula- 
tion time and the user expertise needed to use these techniques do not match the re- 
quirements of system-level three-dimensional multibody software. In order to develop 
a novel method for gear contact simulation to be included in a commercial multibody 
solver, the following decisions have been taken based on the available state of the art: 
- Contact detection: Hyperreduction for contact detection is still in its infancy and 

needs further development. For this reason the computational performances are 

improved by using geometrical considerations that are related to the specific ap- 
plication field of drivetrain dynamics. 

— Dynamic flexibility: The majority of the applications that involve dynamic sim- 
ulations must include the modal behavior of the full drivetrain but the eigen- 
frequencies related to gears bodies and teeth themselves are outside of the fre- 
quency range of interest for many applications. From this point of view, it was 
decided to concentrate on a correct representation of the contact stiffness to prop- 
erly represent the quasi-static behavior of the gear contact and the overall three- 
dimensional system-level dynamics. The accurate evaluation of the contact stiff- 
ness is of paramount importance in the definition of the dynamic modes of the 
full drivetrain. 

- Contact stiffness formulation: The contact interactions of finite element meshes 
require a very fine spatial discretization to properly capture the correct Hertzian 
nonlinear behavior during contact. For this reason it was decided to focus the 
attention on the development of a method that combines the advantages of both 
general finite element formulations but exploits analytical formulas near the con- 
tact regions. 


While the first point is important but out of the scope of this chapter, the second and 
third bullets highlight how available techniques that can be found in the literature 
[3] have been enhanced thanks to a cooperation between Siemens PLM Software and 
the Mechanical Engineering Department of KU Leuven (Section 12.4.2). These devel- 
opments lead to a method based on MOR [102, 12, 16] that allows to describe in an 
accurate way the gear contact stiffness, including effects such as gear body deforma- 
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Figure 12.14: Finite element mesh of a spur gear. 


tion, Hertzian nonlinear stiffness, and teeth convective couplings, while remaining 

extremely efficient. The reasons for the efficiency and accuracy of the method are: 

— Efficiency: The problem is treated quasi-statically; thanks to the MOR technique, 
very few degrees of freedom are retained to describe the teeth deformation. When 
possible, potential symmetries in the gears geometry are also exploited for effi- 
ciency purposes. 

- Accuracy: Despite the application of MOR, the contact interaction is statically 
quasi-exact with respect to the full-order finite element model. Convective de- 
formation terms that couple the deformation of different teeth are accurately 
retained. Local dynamic effects such as teeth dynamic vibrations that are less 
often of relevance during standard operations are instead discarded. While these 
dynamic effects might be relevant for problems such as dynamic ring gears ex- 
citations and high-speed applications, the method is implemented in a modular 
way so that future extensions are efficient to implement. 


12.8.3 User-related challenges 


The developed technology based on MOR achieves the objectives targeted at the be- 
ginning of the development but a key component still needs to be addressed to al- 
low a smooth user experience and limit the amount of expertise needed to use the 
method: usability (Section 12.4). For this reason the parameters that control the level 
of static completeness of the reduction space can be adjusted with a single parame- 
ter that ranges between zero and one where the limit value of one represents exact 
static completeness at the expense of some longer preprocessing time and slightly 
slower simulations while lower values allow to obtain a trade-off between accuracy 
and speed. Moreover the user is provided with a parametric mesher that automatically 
creates the gears finite element meshes based on a few parameters (Figure 12.14) that 
is used for the automatic generation of the reduction space while further minimizing 
the user intervention. 

The interfaces between the MOR method proposed and the multibody solver are 
integrated into a user-friendly application-driven user interface — the Simcenter 3D 
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Transmission Builder (Figure 12.15) — that proposes also a simplified work flow for 
the creation of complex drivetrains. Practically, thanks to the dedicated Simcenter 3D 
Transmission Builder interface and application-specific choices related to the MOR 
technique it was possible to obtain a seamless and user-friendly usage of advanced 
MOR numerical techniques available to nonexpert users. 
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Figure 12.15: Simcenter 3D Motion — Transmission Builder. 


12.8.4 Validation of MOR for drivetrains against experimental 
results 


The described numerical method based on MOR is released as a product in Simcen- 
ter 3D Motion under the name of Advanced-FE preprocessor gear contact. Before the 
product release, given both the complexity of the method and the number of assump- 
tions made during development, the methodology has been validated using multiple 
numerical and experimental results. In this chapter we present a subset of the exper- 
imental validation results to show the accuracy of the proposed method. For a larger 
set of examples we refer to [85]. The validation process has been carried out thanks to 
the usage of an in-house precision gear test rig [74] jointly developed by Siemens PLM 
Software, KU Leuven, and the University of Calabria. The test rig has been designed 
and manufactured to assess typical gear-related physical quantities in static and dy- 
namic conditions, under imposed conditions of misalignment and shaft compliances. 
Particular attention is given to the measurement of gear pair transmission error (TE) 
[75], which is a typical key performance indicator used for the assessment of drivetrain 
NVH performances. 
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As an illustrative example we present validation results related to the complex 
case of gear contact between two spur gears, including large friction, microgeome- 
try modifications, and different loading conditions. The measured TE is shown in Fig- 
ure 12.16. It can be seen that despite the wide range of torques applied, the proposed 
method is able to match the TE with a high degree of accuracy. This is particularly 
striking since the effects of microgeometry, friction (as noticeable in the discontinu- 
ous jumps in the TE), teeth flexibility, and local contact nonlinearities are highly inter- 
acting with each other. The experimental results and multiple numerical validations 
carried out confirmed the good performances of the method in terms of both accuracy 
and speed. 
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Figure 12.16: Experimental-numerical comparison of the proposed approach for TE evaluation. 


12.9 Use case - lifetime analysis 


In this section, we consider lifetime analysis for railway axles as an example for a 
digital twin in the design phase. The engineer would like to know already in the design 
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phase how lifetime depends on the inspection scheme, the inspection interval, the 
type of load, and the fatigue crack growth. Stochastic MOR serves as a key element for 
the realization of this digital twin. 


12.9.1 Efficient computation of failure probabilities 


An important part of lifetime analysis is the computation of small failure probabil- 
ities, which is a challenge for practical problems with CPU time-intensive function 
calls. As already pointed out in Chapter 10 of this volume of Model order reduction, 
Section 10.2.3, failure probabilities are described by multidimensional probabilistic 
integrals which may be discretized by a multivariate quadrature rule. In the case of 
failure probabilities, the integrand of the probabilistic integral is discontinuous such 
that common quadrature rules will not provide sufficient accuracy. Here the key idea 
is to reformulate the integral into an integral with a smooth integrand and then to ap- 
ply the unscented Kalman filter (UKF) [51] for evaluating the integral. The reliability 
of a product, system, or process is often indicated by a failure function: 


<0 unsafe, 


12.22 
>0 safe, ( ) 


s(x) = | 


where x is the vector of stochastic variables. The failure function g(x) describes a dam- 

age mechanism, e. g., 

1. the mechanical stress or the temperature exceeds a given threshold (finite element 
analysis, computational fluid dynamics); 

2. the amplitude of oscillations exceeds a given threshold (linear and nonlinear 
modal frequency analysis); 

3. changes of the microstructure of the material as a prestage of cracks differs from 
a given pattern (stochastic Voronoi techniques, finite element method); 
the crack size exceeds a given length (finite element method + crack size analysis); 

5. achemical species exceeds a given concentration (computational fluid dynamics, 
ChemKin). 


Using this failure function, the failure probability reads 


P(g(x) < 0) = | Px(x)dx, (12.23) 
g(x)<O 


where p(X) is the stochastic density of x. We restrict our presentation to the case of 
independent standard normally distributed variables x = (x4, ... re ge In the general 
case of a failure function depending on nonnormally distributed variables x;, e. g., 
the Rosenblatt transformation may be applied [90], mapping x; to x; fori = 1,...,n. In 
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order to obtain an integral over R”, an indicator function is introduced, 


P(g(x) < 0) = | T,(X)p,(x)dx, (12.24) 
R” 
where 
_f0 g&x)>0, 
T(x) = | 1 g(x) <0. (12.25) 


For practical applications, the Monte Carlo method is too expensive to evaluate the 
integral in (12.24) because of the high computing time. So a stochastic MOR method 
is required. A standard method for approximation of the integral in (12.24) is the first- 
order reliability method (FORM) [48, 47]. This method first computes the so-called beta 
point (the point of highest failure probability) and then constructs a linear approxima- 
tion of the failure function in the beta point. For highly nonlinear failure functions g, 
the failure probability of the FORM will not be accurate. An extension of the FORM, 
the second-order reliability method [14], is more accurate but requires second-order 
derivatives, which are often not available in practical applications. Because of the dis- 
continuous integrand in (12.24), standard quadrature formulas will in general lead to 
bad approximation properties. Our stochastic model order method now consists of a 
reformulation of (12.24) in an integral with a continuous integrand and subsequent 
application of a nonlinear filter method. The reformulation is possible if the failure 
function g(x) is continuously differentiable in its coordinates and strictly monotone 
in at least one coordinate (backmapping approach); see [110]. Without loss of gener- 
ality, let g(x) be monotone in x,. It follows that the critical x, defining the limit state 
can be expressed as a function of x,,...,X;_1: 


Xn F CX EES ,Xn-1) (12.26) 
0 =8(X.. -> Xn- Xn). 
Then the failure integral (12.24) reads 


P(g(x) < 0) 


z | PiX): Prn-1Xn-1) | onnda.dxn 


Ret 


= | P(X) ++ Pn- Mp AO, - - - Xp_1) dX... Xy_-1, (12.27) 


Rei 


with 


h(t... Xq-4) = serfe( SOx, ae | 
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The smoothness of h follows from the implicit function theorem. The failure proba- 
bility P(g(x) < 0) can thus be interpreted as the mean of the smooth function h. For 
evaluation of this mean a nonlinear filter method, called UKF [51], is applied. This fil- 
ter can be used to estimate the mean and covariance of a nonlinear stochastic process 
f(w), where w € R”™ is a normally distributed random vector with mean E(w) and 
covariance P™” € R™*", So-called sigma points ¥®, together with weights wm" 
and W;°”, are constructed and mapped to Z © = f(x®) fori = 0,...,p. The unscented 
filter then yields an approximation of the mean y and covariance P” of the nonlinear 
function by 


P . 
E(z) x X AA 
i=0 


By Taylor expansion one can show second-order accuracy of mean and covariance 
[51]. 


12.9.2 Stochastic crack growth 


In stochastic crack growth, the crack depth a is a function of the stochastic parameter 
vector x and load cycle N, 


a = a(x, N), (12.28) 
and the failure function (12.22) is given by 
g(x, N) = agit — A(X, N), (12.29) 


where ait denotes a critical crack depth indicating failure of the component. We con- 
sider elliptical surface cracks given by crack depth a and crack form b = a/c. Both 
the initial crack depth a = ag and the initial crack form bọ are stochastic. Further 
stochastic parameters are given by the so-called POD curve and the probability of 
crack initiation. In contrast to most other places in this handbook, here “POD” does 
not mean “proper orthogonal decomposition,” but “probability of detection.” The POD 
curve gives the probability of crack detection during inspection and so characterizes 
the inspection scheme. The final crack depth monotonically depends on the initial 
crack size ag such that the reformulation of the failure integral of Section 12.9.1 can be 
applied with x, = do in (12.26). The goal is to compute the cumulative failure proba- 
bility for a given number of equidistant inspection intervals, under consideration of: 

— the replacement of a component if a crack is detected during inspection; 

— the probability of crack initiation (input). 
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We use the following notation: 


T; i-th inspection time 

[amin Imax] domain of definition of crack depth a 

a initial crack depth 

B minimum detectable crack size at time T% 
(according to POD curve) 

[Bmin> Bmax! domain of definition of £ 

x vector with realizations of all stochastic variables 
except for a and f 

Qn critical initial crack size leading to failure at time T,,, 
A(X, Am Tn) = Aerit 

P; probability that the crack is not detected during 
inspections and reaches the critical crack depth 
at time T, 

P3 probability that the crack does not exceed the critical 
depth and is detected at time T, 

P: cumulative failure probability at time T, 
under consideration of failure of replaced components 

Gh probability of crack initiation in interval [T,,_), T,] 

Px Pa Pp stochastic densities of x, a, f 


The probability of detection of a crack with depth a is given by 


a 
Bain 
The probability of detection of a crack at time T,, is 
Tg = Ig, a) = Ip(a(X, a, Tp). 


Probabilities P} and Pr are given by 


P= [1iipcat, P= | Hapea, (12.30) 
Q Q 

with 

oe: [ee TpPada forn = 1, 
Ty) = z 

je G-p A-I pada forn>1 

and 

Amax 
cane fo Pada forn=1, 
I(x) = 
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The cumulative failure probability can then be computed by the following recursive 
scheme: 


ae ae (12.31) 
PET = PEC, + +++ + Cuan) + PRC Hoe + Cn) Ho PRC 
(Pl) Pt (12.32) 


+ (cP + CP4) pee 


++ 


+ (c,P} + oP’ +e CyP 4) PL. 


The integrals in (12.30) are the mean values of I (x) and I H (x) and are evaluated by UKF 
as previously described. The critical initial crack depths &„-1, &„ appearing as integral 
limits in the definition of I H (x) and I; (x) are computed by a bisection algorithm. This 
procedure of evaluating the failure integrals has been validated by Monte Carlo for a 
model problem in [73]. 


12.9.3 Lifetime of railway axles 


For lifetime investigation of railway axles, we use the failure function in (12.29) with 
crit = 10 mm. The stochastic distributions of the initial depth ag and initial form bp of 
the elliptical surface crack are given in Table 12.1. In this study two inspection schemes 
are considered: 

— ultrasound far end scan; 

— ultrasound mechanized. 


In the first case the axle is scanned by sound from one end of the shaft to the other in 
the longitudinal direction, in the second case in the radial direction. The POD curves of 
these schemes are shown in Figure 12.17. The cumulative density function of crack ini- 
tiation is given later together with the results of the results of lifetime analysis. Fracture 
mechanics for railway axles are subject of current research [54, 108]. Here the fracture 
mechanical simulations are accomplished by the simulation program ERWIN from the 


Table 12.1: Stochastic parameters of crack growth. 


Type of distribution Meaning 


bọ Uniform distribution on [0.6, 1] Crack form: quotient of crack depth and length 
(bo = a/c) 
ay Shifted exponential distribution Initial crack depth 
with A = 100 and shift 0.012 
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POD curves 


= UT mechanized 


m= UT far end scan 


Probability of detection 


Figure 12.17: POD curves. 


Fraunhofer Institute IWM in Freiburg, Germany [34]. It predicts the propagation of el- 
liptical surface cracks, for different types of bendings. The stress intensity factors are 
represented by so-called polynomial influence factors [15]. Inputs of the crack sim- 
ulation are internal and external stress profiles (due to the external load spectrum), 
da/dN-curves, and the initial crack depth and form. 

It should be noted that we call the crack simulation a black box. Inputs are the ini- 
tial crack depth and length, and output is the final crack depth, which is subsequently 
used for evaluation of (12.29). 


Table 12.2: Test cases. 


Test cases Typeofload Inspectionscheme Inspection interval 


Case 1 Partial load UT far end scan 100,000 km 
Case 2 Full load UT far end scan 100,000 km 
Case 3 Full load UT far end scan 50,000 km 


Case 4 Full load UT mechanized 100,000 km 


Four test cases are considered with different loads, inspection schemes, and inspec- 
tion intervals (Table 12.2). The resulting lifetimes are shown in Figure 12.18. Figure 12.18 
also shows the probability of crack initiation, which is input to the lifetime calcu- 
lations. Lifetime is represented as a function of the deferred distance in kilometers, 
where one kilometer corresponds to 354 load cycles. The results show how lifetime 
depends on the probability of crack initiation, the type of load, and the inspection 
scheme. As expected, the full load case with the worst inspection scheme and inspec- 
tion interval 100,000 km (case 2) has the shortest lifetime. The second worst is the 
partial load case (case 1) with the same inspection scheme and interval. Lifetime of 
the full load case can be improved by either switching to shorter inspection intervals 


12 Model order reduction and digitaltwins —— 419 


Lifetime for different scenarios 


0,08 
=== Probability of crack 

£ 0.06 initiation (input) 
£ Case 1 
° 
> 0,04 
= 
Z ——Case 2 
£ 
© 0,02 
a — Case 3 

0o -= 

0.£+00 Case 4 


Figure 12.18: Lifetime for different scenarios, with specified probability of crack initiation (black 
dotted curve). 


(case 3) or to a better inspection scheme (case 4). Both cases lead to longer lifetimes 
than cases 1 and 2, where shorter inspection intervals are more effective than a better 
inspection scheme. One call of the crack simulation program takes approximately 15 
CPU seconds on a 3.2 GHz processor. For each curve shown, 3,000 to 4,000 simulation 
calls are required. 


12.9.4 Conclusion 


As an example of a digital twin in the design phase, lifetime analysis for railway axles 
has been presented, for different inspection and load case scenarios. By using adapted 
stochastic MOR methods, stochastic crack growth under consideration of inspections 
can be computed in reasonable computational time, which would not be possible with 
pure Monte Carlo methods. Key elements are the reformulation of the failure integrals 
as mean values of continuous integrands so that a nonlinear filter like the UKF can be 
applied with sufficient accuracy. 


12.10 Use case - circuit simulation 


12.10.1 MOR in the electronics industry 


MOR has been part of the standard techniques used in circuit simulation for a long 
time, with publications dating back to at least 1990 [78]. The relation is bidirectional, 
with the circuit simulation and semi-conductor industry providing several benchmark 
cases [92, 88]. It is not only Moore’s law [70], which states that circuit design complex- 
ity roughly doubles every 2 years, that drives this relationship; it is also the growing 
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need of circuit designers to include more physical details in their simulations and at 
the same time to have control over accuracy and performance. From a technological 
point of view, MOR methods will have to be developed that can deal with the growing 
complexity: The number of unknowns typically increases with the size of the design 
and the advance of the technology node (decrease of transistor dimensions), while 
the (Jacobian) density of the problem typically increases with the amount of detail in- 
cluded (wire resistance, capacitive coupling, inductive coupling). The MOR challenge 
lies hence not only in the problem’s dimension, but also in the problem’s complexity 
in terms of coupling and detail. From a business point of view, it is clear that scalabil- 
ity of simulation software is key (Section 12.2). What is less clear, however, is in which 
part of the flow the scalability should apply. For instance, before actually simulating a 
circuit, the differential algebraic equations describing the behavior of the circuit first 
need to be constructed. This process is called extraction and the resulting description 
of the circuit that can be translated into a system of differential algebraic equations is 
called netlist. Whether to apply MOR during this extraction phase and/or the simula- 
tion phase is not always clear, not only for reasons of robustness and reliability, but 
also for commercial reasons. In the remainder of this section we will focus mainly on 
the technical challenges. 


12.10.2 Technological challenges 


At first sight, MOR problems arising in circuit simulation may seem easy as they fall 
into the most elementary class of linear time-invariant dynamical systems. Electrical 
circuits that include nonlinear elements such as CMOS transistors are described by 
systems of differential algebraic equations of the form 


dq(x) 
dt 


j(x) + = s(t), 

with node voltages and currents x(t) € R”, (non)linear vector-valued q(t, x), j(t, Xx) € 
R” with the electrical branch contributions, and sources s(t) € R”. Typically, only a 
linear subsystem is considered for reduction. This linear system models the behavior 
of linear resistors (R) and capacitors (C) and is usually considered in the frequency 
domain:? 


Gv + sCv = Bu, 


with node voltages v € R”, inputs u € R“, Laplace variable s, conductance and capac- 
itance matrices G, C € R®™”, and input mapping B € R™*. One distinguishes between 
internal nodes and terminals (or ports): Internal nodes only have connections to other 


3 For simplicity we do not include inductors (L). 
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nodes via RLC elements, while terminals have also connections to non-RLC elements 
like transistors. This means that internal nodes are candidates for elimination while 
terminals need to be preserved, in general, because they connect the linear subsys- 
tem to the rest of the system. In the circuit simulation community, MOR is also known 
as netlist reduction or parasitic reduction, with the adjective parasitic referring to the 
nonintentional nature of the RLC elements that model the wire resistance and capac- 
itive and inductive coupling. 
Methods from several well-known categories are used for reduction: 
— Krylov subspace projection, among the first of MOR methods to be applied to elec- 
trical circuits [33, 72]; 
- balanced truncation, with a priori error bounds [84, 9]; 
— modal truncation, used for the construction of behavioral models [87]; 
— nodal elimination methods, which have as advantages the existence of error 
bounds and ease of implementation [98, 96]. 


An advantage of nodal elimination methods (and to a lesser extent modal truncation), 
especially in the context of circuit simulation software, is that the ROMs can naturally 
be translated into a reduced circuit with meaningful RLC elements. For Krylov sub- 
space and balanced truncation methods, the ROMs are typically dense with nonphys- 
ical (negative) RLC elements, and integration requires interfaces to deal with matrix- 
based circuit descriptions. 

Despite the developments in the MOR domain, even the problem of reduction of 
linear circuits is still not considered as solved. The following key challenges can be 
identified for linear circuits: 

— Linear solve costs: For subcircuits that contain only resistors or capacitors, 
projection- and elimination-based MOR procedures are error-free [89, 98], but 
the question of how to minimize the fill-in created by node elimination for the full 
design system matrix factors is still open (and becomes more difficult for mixed 
RLC circuits). 

— Coupled problems: With decreasing feature sizes and increasing frequencies, ca- 
pacitive and inductive coupling becomes stronger and denser. As a result, the orig- 
inal system matrices become denser, the reduction procedure becomes more ex- 
pensive, and the ROM may become even denser, rendering MOR less effective. 

— Precise accuracy performance tuning: For users it is important to be able to trade 
off between accuracy and performance. For instance, for top-level verification, 
one can (and often has to) accept less accuracy in order to improve simulation 
speed or to make simulation possible at all. The challenge is here twofold: (1) how 
to estimate the effect on accuracy when integrating the reduced circuit into the full 
design and (2) how to estimate the effect on the overall simulation time. 

— Which method to apply when: There is not a single best method for MOR that fits 
all problems. Hence there is a need to be able to select automatically and dynam- 
ically, based on certain characteristics, which method to use. 
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—  Variability-aware analysis: Uncertainty quantification of the impact of process 
variability on design robustness requires ROMs that are valid for ranges of design 
parameters (with as additional complication that there can be many parameters). 

— With designs having a growing number of nonlinear devices like CMOS transis- 
tors, also the need for robust and efficient MOR methods for nonlinear systems 
increases. 


12.10.3 SRAM memories: critical path simulations 


SRAM memory designs [109] are of interest for MOR methods for several reasons. 
SRAM designs typically have a relatively large memory block of 6- or 8-transistor 
bitcells (depending on the memory size) and some smaller control blocks. Although 
designers often replace the large bitcell matrix by a much smaller model (manually), 
for top-level simulations an automatic, accuracy-preserving method is required. Fur- 
thermore, the extraction (modeling) of the many and long wordlines and bitlines may 
result in netlists with not only many resistors but also many coupling capacitors. Espe- 
cially for so-called critical path simulations, where one wants to ensure that the delay 
for read and write operations is within specifications, reduction must be done with 
care to guarantee that the delay error is within picoseconds or even less. Additionally, 
to assess robustness of the design against process variations, one needs to run many 
simulations and hence simulation time needs to be minimized. In short, for memory 
designs, MOR has to deal with all the challenges mentioned in the previous section. 

In Figure 12.19 we show the results for time-domain simulations with reduction 
settings varying from conservative to aggressive. The main impact on accuracy (error 
in delay) and performance (simulation time) is caused by how coupling capacitors are 
reduced. It depends on the type of verification how much error is acceptable: This can 
vary from tens of picoseconds to less than one picosecond. The results are produced 
using the circuit simulator Eldo Premier [68]. 


12.10.4 Concluding remarks 


Driven by rapidly increasing design sizes and complexity, MOR, also known as para- 
sitic or netlist reduction, has become a standard and indispensable option in modern 
circuit simulators. Although current MOR methods are suitable for robust accuracy 
performance control of simulation of advanced CMOS designs, more advanced CMOS 
nodes and verification requirements will require the development of new methods and 
approaches. Not only accuracy and performance remain key priority, also methods 
that can be used in the context of other applications, such as variability-aware de- 
sign, are required. In particular parameterized MOR methods for linear and nonlinear 
systems will need to be further developed in order to make them suitable for use in 
future industrial software (Section 12.2). 
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Figure 12.19: Voltage on vertical axis between —100 mV and 800 mV. Time on the horizontal axis 
between 150.125 ns and 150.180 ns. Time-domain simulation results of Eldo Premier [68] for conser- 
vative (orange dashed) and aggressive (blue dashed-dotted) reduction settings, compared to the 
golden reference (green solid). The simulation with aggressive reduction settings is two times faster 
than the simulation with conservative settings, but one has to pay with less accurate results (which 
however may still be acceptable depending on the type of verification): The delay error in the signal 
increases from less than 1 ps to almost 3 ps. 


12.11 Conclusions and outlook 


Within this contribution, we have reviewed a number of industrial success stories 
of MOR in the context of digital twins (Sections 12.5-12.10). Through MOR the corre- 
sponding simulations could be accelerated and reduced in their memory footprint. 
This enabled novel applications which would not have been possible without these 
improvements. Therefore, MOR is a key enabler for a new generation of digital twins 
(Sections 12.2 and 12.3). With respect to sustainable industrial applications and com- 
mercial software packages containing MOR engines it is crucial to close the gap from 
algorithms to products. Here, professional software development plays a crucial role, 
which we have addressed in Section 12.4. 

MOR allows to reduce computational execution time of models while controlling 
accuracy. Application- and purpose-specific models with different requirements in 
terms of speed and accuracy can be realized. At the same time, MOR liberates sim- 
ulation models form their execution engines, i. e., their specific simulation tools and 
numerical solvers. This allows a separation of the creator of a digital twin — typically 
a simulation engineer — and the consumer — anyone downstream including machines 
themselves — through appropriate application interfaces. Furthermore, this allows not 
only to reuse the models during operation as highlighted by some of the use cases, 
such as virtual sensors (Section 12.5), but also novel licenses and business models 
[37], such as pay-per-execution time. In particular, realizing a pay-per-execution busi- 
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ness model during operation allows to scale with the number of products sold by a 
company assuming that each product contains a digital twin. Typical business mod- 
els in the context of simulation tools only scale with the number of engineers working 
in a company, assuming that each engineer uses corresponding tools. Therefore, the 
impact does not only lead to novel application areas but also to the way how industrial 
value streams are organized. 

Summarizing the current rapid advancements in MOR, a novel generation of digi- 
tal twins, so-called executable digital twins [45], is likely to emerge in the near future. 
An executable digital twin is a specific encapsulated realization of a digital twin with 
its execution engines. As such they enable the reuse of simulation models outside R&D 
departments. In order to do so, the executable digital twin needs to be prepared suit- 
ably for a specific application out of existing data and models. In particular, it must 
have the right accuracy and speed. The executable digital twin can be instantiated on 
edge devices, on premise servers, or in cloud environments and used autonomously 
by anonexpert or a machine through a limited set of specific application programming 
interfaces (APIs). 

In order to realize this vision, several key challenges remain open though many 
of them are subject to active research efforts: 

- How to prevent virtual reverse engineering? Thanks to fast execution times, 
digital twins could be executed many times allowing to reverse engineer specific 
features, e. g., optimal control logics. 

— How to leverage MOR with black-box solvers? Many commercial simulation 
tools do not provide APIs for systematic interaction with their kernels. However, 
this is a central requirement for integrating novel MOR tools. At the same time the 
development of such APIs will take significant time due to the existing develop- 
ment processes. That is, first, such APIs must be ranked high enough in feature 
backlogs for next software releases, and second, these features need to be vali- 
dated and verified before they are available. 

- How to provide certifiable accuracy bounds for ROMs? The usage of MOR to 
enhance operations, e. g., in the context of model predictive control, requires cer- 
tifiable models, e. g., ensuring conservation of important quantities. 

- Howto combine/integrate machine learning and MOR technologies better? 
Machine learning technologies, e. g., neural networks, are rapidly expanding in 
industrial applications addressing similar aspects as MOR. However, combined 
concepts are still missing. 

— Howto package ROMs appropriately? Even though containerization technolo- 
gies, e. g., Docker, have matured over the last years, it is not clear how to leverage 
them in the context of MOR, e. g., appropriate interfaces and standards are miss- 
ing. 
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Mathematical research in MOR as well as close collaboration with industrial software 
providers and users will be key to address these challenges and ultimately realize the 
vision of executable digital twins. 
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13 MOR software 


Abstract: This chapter is devoted to an important requirement of successful model or- 
der reduction (MOR) application, namely, the software aspect. The most common sit- 
uation is the existence of a so-called full model, i. e., a high-fidelity, high-dimensional 
simulation model, that needs to be accelerated by MOR techniques, optimally without 
reimplementing the partially complex reduction techniques, as presented in the first 
volume of this handbook. 

Initially, as neither full simulation models nor MOR algorithms are to be repro- 
grammed, but ideally are reused from existing implementations, we concentrate on 
the aspect of the interplay of such packages. We will discriminate, discuss, and exem- 
plify different levels of solver “intrusiveness” that allow corresponding reduction tech- 
niques to be applied. On the one hand, most effective MOR techniques require deep 
access into the full model’s simulation code. On the other hand, application-specific 
full model simulators may only offer very restricted access to internals, especially in 
case of commercial packages. This gap in requirements and practical accessibility mo- 
tivates the discrimination into “white-box,” “gray-box,” and “black-box” simulation 
scenarios. In particular, we exemplify the ideal case of MOR for white-box situations 
on two examples, namely, parametric linear elliptic PDE and parametric nonlinear 
ODE systems. Depending on those access classes, different corresponding reduction 
techniques can be applied. 

The second part of the current chapter then discusses existing MOR software. Sev- 
eral program packages exist which provide MOR techniques. They differ in availabil- 
ity, licensing, programming language, system types, physical application domains, 
external simulator bindings, etc. We give an overview of the most relevant of those 
MOR packages, such that applicants can identify potential suitable software library 
candidates. 
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13.1 Introduction 


Following the main motivation of this handbook, we recall that model order reduction 
(MOR) is a key technique to enable high-level simulation scenarios. By reducing the di- 
mensionality or order of a high-fidelity or “full” model, the generation of a “reduced” 
model is expected to give approximate but still accurate results while decreasing run- 
time, memory, and ideally financial costs for simulation and product development. 
By reduced models, more complex simulation settings are possible beyond single for- 
ward solves: “Multiquery” sampling is possible enabling uncertainty quantification or 
surrogate-based optimization, and “real-time” response is ideally achieved enabling 
interactive design or real-time control applications. We want to refer to the multitude 
of textbooks that have appeared during the last two decades such as [35, 4, 2, 30]. The 
key of efficient model reduction is an “offline-online decomposition,” which relates 
to the separation of the MOR into two phases: The reduced model construction phase 
(possibly computationally intensive, performed only once) is denoted “offline phase,” 
while the execution of the reduced model (ideally computationally cheap, performed 
multiple times) is very fast due to small memory and computational demands. 

In this chapter we want to discuss from a rather general level some perspectives 
on the state and perspectives of the interplay between high-fidelity simulation soft- 
ware packages and MOR algorithms and code. Currently, many commercial simulation 
software packages exist, but few offer MOR technology or access to required internals. 
By this, such existing software packages cannot straightforwardly be combined with 
MOR and can thus not be used for modern simulation tasks, potentially representing 
an economic disadvantage. 

On the other hand, MOR researchers have a high interest to make use of these 
commercial packages due to efficiency, accessibility to industrial-sized relevant ap- 
plications, etc. As a result, MOR researchers frequently struggle or fail to get their al- 
gorithms to work with such commercial tools. Typically, they then either are content 
with “toy examples,” and implement the high-dimensional solvers by themselves, or 
try to use the existing commercial packages by workarounds or “hacks.” The ideal 
way, though, of developing elegant and goal-oriented interface access between solvers 
and MOR technology is rarely realized. 

The ideal and frequent goal of MOR is to have reduced models that are compu- 
tationally independent of the full dimension during the online phase. This is called 
“ideal online efficiency.” In order to obtain this, a suitable offline-online decompo- 
sition must realize suitable interaction between the full-order and the reduced-order 
model. In particular for obtaining ideal online efficiency, MOR techniques are typically 
very code-intrusive in the sense that specific details of the full model must be accessed 
in an efficient manner. If a full simulator provides all of this necessary information to 
the outside by suitable functionality, we call such reduction scenario “white-box,” as 
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from an MOR viewpoint the full model is fully transparent. As mentioned earlier, high- 
fidelity simulation packages are frequently not fully capable to provide the required 
ingredients, because of “closed-source” policies, code inaccessibility, code complex- 
ity, or algorithm strategy. For instance, if a full model performs discretizations in a 
matrix-free fashion by full mesh traverse, or a package only can assemble full finite 
element matrices, then a rapid partial evaluation of discretization operators or single 
matrix entries is difficult without (costly) traversing the mesh or assembling the global 
matrix, which is relevant for many MOR procedures. 

In particular, beside the clean, transparent, and optimal white-box scenarios 
mentioned earlier, we want to introduce notions of “gray-box” and “black-box” 
scenarios, where the high-fidelity model provides partial or none of the required 
information. For such situations, there exist several practical solution strategies, 
workarounds, or “hacks” to enable MOR without full access to required ingredients. 
We will particularly also address such strategies. 

As a general observation, the tedious and time-intensive development of software 
is not warranted suitable acknowledgement from the scientific communities in ap- 
plied sciences. Regrettably, this is always just expected and accepted as a by-product 
of some main disciplinary scientific work/progress. This is reflected in this chapter, as 
we will not have many references to journal articles but mostly students theses, pro- 
ceedings articles, presentations, or PhD theses that are partially devoted to the aspect 
of MOR software. 

By nature, many reduction methods will be mentioned in this chapter, and most 
of these techniques are described in detail in Volumes 1 and 2 of the current handbook. 
In order to avoid an abundant reference list, at first use of the methods, we solely refer 
to the corresponding chapters. 

This contribution should serve both applicants as well as MOR researchers: Appli- 
cants can identify which kind of internals they could or are willing to provide for the 
most elementary reduction techniques. Correspondingly, they can decide about which 
software packages to use and which “hacks” exist to enable MOR for their problem. 
Scientists working on MOR software should get an incentive to extend or contribute to 
existing academic MOR software packages. 

This chapter is structured as follows. In the next section, we exemplify the inter- 
play between full simulation packages and MOR algorithms. For this we choose two 
model examples that in certain sense represent different “corners” in the space of MOR 
model problems. The interplay of full and MOR software packages needs to be real- 
ized for white-box and gray-box, as well as black-box scenarios, and we comment on 
some main coupling techniques. In Section 13.3 we then give an overview of existing 
MOR software packages, both toolboxes of existing commercial simulator packages 
and packages developed at academic institutions. We give short characterizations of 
the packages such that users have some guideline on which package to choose in their 
respective application. We conclude in Section 13.4 with a summary and some recom- 
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mendations for both commercial simulator developers/companies and academic MOR 
researchers. 


13.2 Interplay of full and reduced solvers 


In order to clarify and illustrate the needs and challenges in interplay of MOR software 
with full solvers, we remain conceptional in this section, but still are very specific in 
terms of two model types. We address both the reduced basis (RB) methods commu- 
nity, aiming at solving parameterized partial differential equations (PDEs) (Chapters 1, 
4, and 6 in Volume 2 of Model order reduction), and control theory researchers, aiming 
at ordinary differential equation (ODE) systems. Therefore, we choose two model ex- 
amples representing those fields. For those models we exemplify each a plain vanilla 
state-of-the-art reduction scheme, which despite simplicity is intrusive in the sense 
that it requires deep access into the full solver. Therefore, we denote these as white- 
box approaches. Correspondingly, we exemplify how, in realistic cases with limited 
information about or restricted access to the full model, MOR can still be realized in 
gray-box or black-box scenarios. 

As our first model we select a parametric variational form, e. g., as appearing by 
finite element discretizations of a linear elliptic PDE. As our second model we use a 
nonlinear ODE system. This covers both a linear and a nonlinear, a steady and an 
unsteady, and a parametric and a nonparametric case. 

For these models and reduction scenarios, we necessarily must repeat some no- 
tation and terminology which appears at various places within this handbook, but 
which we consider essential to make the point. In the last subsection, we comment on 
possible coupling techniques of full and reduced solver packages. Readers who are fa- 
miliar with those concepts or who are primarily interested in MOR software packages 
may skip this section and directly continue with Section 13.3. 


13.2.1 Model 1: parametric stationary variational problem 


We assume to have a Hilbert space X of functions, which is assumed to be finite- 
dimensional of high dimension n and spanned by basis functions y,,i = 1,...,n. We 
want to solve a linear parametric variational problem as is typical in RB methods 
(Chapters 1, 4, and 6 in Volume 2 of Model order reduction or [14, 17]), omitting the out- 
put for ease of presentation: For a parameter u € P from a parameter domain P c RP, 
find a solution u(u) € X such that 


alu), v;y) =f(vsy), veX. (13.1) 
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Here, for any py, the form a(.,-;) : X x X —> Ris bilinear and f (; u) : X > Ris linear. 
Introducing the system matrix and right-hand side 


AW) = (app pE D) ER", £90 = FEN) € R", (13.2) 


the solution is obtained by solving the corresponding linear system for the degree of 
freedom vector x(u) = CAH € R” and subsequent linear combination 


AQW)x() = fu), u) = È xapi (13.3) 


i=1 


The problem is typically assumed to be solvable, e. g., by ellipticity or inf-sup stability 
assumptions. Note that by choosing X = R”, y; being the standard basis, we obtain 
u(u) = x(u) and hence this model formulation also simply covers parametric linear 
equation systems of the form (13.3) without the need for a variational form (although 
such can easily be constructed [14]). 

To enable efficient model reduction, the parametric dependency is assumed to 
be based on parameter separable forms (“affine assumption”), i. e., there exist Q, co- 
efficient functions 64 : P > R,q = 1,...,Q,, which can be rapidly evaluated, and 
parameter-independent system components A? € IR" such that 


Qa 
Alu) = È 04(u)A4 
q=1 


and a similar expansion for f using coefficients of (u) and components ff, q = 1,..., Qp. 
We denote the inner product matrix of the space X as 


K i= (pr Pij- € R”, oa) 


which allows to compute norms (or errors, projections, orthogonalizations, Riesz rep- 


resenters for error estimation, etc.), e. g., |lu(u)Ily = \x(W)Kx(y). 


13.2.1.1 White-box reduction scenario 


In parametric problems we discriminate between the offline phase (reduced model 
construction, potentially computationally expensive and involving full system compo- 
nents or solves) and the online phase (assembly and solve of reduced system, rapidly 
executable). The latter can then be evaluated in many-query contexts. As reduction 
technique we consider Galerkin projection. For this, in the offline phase, a matrix 
V e R”™*’ is constructed, wherer « nindicates the reduced dimension. In RB methods, 
V can be obtained by various techniques, e. g., as concatenation of solution snapshots 
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(Lagrangian basis), Gram-Schmidt orthogonalization of such, proper orthogonal de- 
composition (POD) (Chapter 2 in Volume 2 of Model order reduction), or greedy proce- 
dures. All of those have in common that several full system solves (for the snapshots) 
must be executed and, as mentioned earlier, the inner product matrix K may be re- 
quired for orthogonalization. 
After basis matrix generation, the full system components are projected, 
Al := V'AIV e R”, fl:=V' fe R’ (13.5) 


r 


for q ranging from 1 to Q, and Q,, respectively. This concludes the offline step. 

Then in the online step, for any given parameter u € P only the coefficient func- 
tions need to be evaluated and the reduced system is assembled by linear combina- 
tion, 


Qa Qr 
A,(u) = È 01DA, fw) ODT. (13.6) 
q=1 q=1 


The reduced solution then results via its r-dimensional coefficient vector x, (4) € 
R” from solving 


A, Wx W) = f). 


If reconstruction of the approximate solution is desired (e. g., for visualization, etc.), 
the approximation (u) € R” of x(u) is obtained as 


žu) = Vx, (p). 


By realizing this reduction strategy, an ideal online efficiency is obtained, mean- 
ing that the reduced computations in the online stage do not involve any operations 
of high complexity n, but only of complexity depending on small quantities r, Qa, Qr. 
If a full discretization scheme provides all the required internals to realize the above 
reduction and reduced solution steps, we call it a white-box reduction scenario. 

Figure 13.1 illustrates and summarizes this decomposition and white-box inter- 
play of full-order and reduced-order simulation software. 

Note that instead of providing the full-dimensional matrices K, A’, it can as well 
be sufficient that the solver provides suitable matrix-vector multiplication routines. 
By this, also PDE discretization schemes that work with matrix-free implementations 
can be used in a white-box fashion. 


13.2.1.2 Gray-box reduction scenarios 


Now, in practice, the full solver package may not give access to all of the details. In par- 
ticular, we are not aware of any commercial simulator package giving explicit access 
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Reduced solver 


Standard: Offline: 

+ Assemble full system parts * Compute solution snapshots 
A(u), f(u) * Generate basis matrix V 

* Solve full system * Compute projected components 
A(u) x(u) = f(u) AS, ft, 


Auxiliary: 
«Assemble inner product matrix K + Assemble reduced system parts 
«Assemble system components A(u), f(u) 
AY, f3 
+ Evaluate coefficient functions 
85,(u), 89H) 
+ Visualize/postprocess x 


Online: 


* Solve reduced system 
A(u) x(u) = fH) 
* Optional: reconstruct approx.: 
XH) =V x(u) 


Figure 13.1: Required functionality and interplay of full (high-dimensional) and reduced solver in 
white-box MOR scenario for stationary variational problems. 


to parametric matrix components and coefficient functions. We thus call this scenar- 
ios gray-box, as the full solver does not give complete insight into the problem, but 
still MOR is desired. 


Missing K 

A first problem may consist in the situation that the full model does not provide an 
inner product matrix (13.4). This prevents measuring errors, computing projections, 
and orthogonalizations in the correct function space norms. 

If geometry information of the underlying mesh and information about the dis- 
crete function spaces (polynomial degree, order of the nodes) is available, this matrix 
may be constructable “by hand” outside of the simulator code. Still this is of the same 
technical complexity as assembling a finite element matrix, which may require con- 
siderable effort. 

Alternatively, instead of working with the correct K, one could choose certain al- 
ternative matrices. This corresponds to choosing another inner product on the solu- 
tion space. A common choice is K := I,,, the identity, i. e., choosing X = R” as standard 
Euclidean space. Note that this approach is simple but may lead to severely subop- 
timal bases as the meaning/weighting of the different entries of the degree of free- 
dom vector is not respected. For example, if those degrees of freedom relate to values 
on grid cells of largely varying size, the small cells are given the same weight as the 
large cells, which may mislead basis generation (e. g., when computing a POD without 
proper weighted inner product) and thus deteriorate the global accuracy. This choice 
is common in engineering practice, but should be applied with care. Another choice 
for the inner product matrix could be K = A(j) for a fixed reference parameter ji if the 
system provides a symmetric, positive definite system matrix. This choice actually cor- 
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responds to the “energy inner product” in case of thermal diffusion problems, which 
is known to be beneficial for error estimation [14]. 


Missing parameter separable decomposition 

Most prominently the parameter separable decomposition may not be available. A first 
modification of the procedure could consist of the following. One can omit to work 
with components A4, ff, and instead directly assemble the reduced system by projec- 
tion of the system matrices during the online phase, 


Au) = V'A(QWV, fru) = VFG). (13.7) 


We remark that the definitions of the reduced system and the parameter separable de- 
composition in the previous subsection exactly yield this equivalent representation 
in the case of parameter separable decomposition. Thus this is no approximation step 
but rather a reformulation of computing the reduced system. However, in (13.7) there 
is a computational bottleneck, as the assembly of the full system A(w) and f(y) is ex- 
pensive, in particular polynomial in n. Hence, the online efficiency is sacrificed. 

A second way of solving the missing parameter separability is to produce an ap- 
proximate parameter separable approximation of the problem. Several approaches 
can be found in the literature that help to solve the issue of the missing separable 
decomposition in the system components. For the right-hand side vector the discrete 
empirical interpolation method (DEIM) [8] can be applied, which will be treated in 
more detail in the context of nonlinear unsteady problems in Section 13.2.2. A variant 
of this procedure has been formulated for matrices, which is called the matrix DEIM 
(MDEIM) [40, 28]. In the online phase this method only requires the evaluation of a 
few matrix entries (A(X)... jn for a set of M “magic index pairs” (imjm) M = 1,...,M. 
Still, this pointwise assembly might not be possible or highly inefficient with a given 
solver package, for instance, if it only can assemble the complete system matrix and 
not single entries. Then obviously, extraction of single matrix entries has complex- 
ity polynomial in n due to the required assembly of the complete matrix. But if this 
local entry assembly is possible in an effective way, then this procedure can be online- 
efficient. Also, the procedure can generate an exact parametric representation: If the 
parametric matrices lie within a finite-dimensional space, this MDEIM procedure will 
find such an exact representation, where M is exactly the dimension of this covering 
finite dimensional space. 

An alternative can be parametric regression or interpolation of system matrices 
[41], denoted as “operator extraction”: Based on a given set of system matrix snap- 
shots, a polynomial approximation or interpolation in the parameters is realized and 
successfully used. In the online phase, access to neither A(u) norits entries is required, 
recovering the ideal online efficiency. However, no error control for assessing the para- 
metric approximation quality is possible by this approach. 
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We want to mention another approach in a PDE context, which was coined the 
“two-grid finite element/reduced basis approach” [7]: One constructs a RB on the given 
(fine) grid in a traditional way, e. g., Lagrangian, POD, or greedy basis. Then in the on- 
line phase, for a new parameter, a full problem is solved but on a coarser mesh. Finally, 
this coarse mesh solution is projected on the obtained (fine mesh) RB. By this, missing 
details on the coarse mesh are potentially included in the projected solution, as such 
fine details are contained in the fine mesh snapshots/RB. The computational cost in 
the online phase clearly is not online-efficient as it depends on the coarse mesh reso- 
lution. But it still can be computationally faster than solving the full problem on the 
fine mesh. In this method, internal details on the geometry, in particular the nesting 
of the two meshes, are required, and details on the discrete function spaces, in order 
to compute the prolongation operator for mapping the coarse mesh solution degree of 
freedom vector to a fine mesh degree of freedom vector, one further needs the inner 
product matrix for projection onto the RB. 

So overall, in the case of the missing parameter separable decomposition gray-box 
scenario, either one sacrifices the ideal online efficiency while maintaining accuracy 
of the reduced model, or one must accept another approximation stage in the reduc- 
tion chain for obtaining the optimal online runtime complexity. 

These are the most common approaches of workarounds or “hacks” around miss- 
ing system information for stationary problems. 


13.2.1.3 Black-box reduction scenario 


We denoted the previous section as gray-box, even if some references rather call their 
approaches black-box. The reason for our nomenclature is that the mentioned ap- 
proaches still require some insight into the discretization or sampling of system com- 
ponents, i. e., knowledge of and access to the system structure. In contrast to this, the 
real notion black-box would be even more restrictive as not allowing any insight into 
the system components or discretization. The most limiting situation would be that 
the system only is observable via input-output pairs (4; X(u;)), i = 1, ... , Maine Then, 
based on these training data, machine learning approaches could be used to infer a 
functional relation between the input parameters and the solution (degree of freedom 
vector), e. g., kernel methods (Chapter 9 in Volume 1 of Model order reduction), or neu- 
ral networks. As a learning of a high-dimensional quantity (degree of freedom vector 
x(u)) may suffer from the curse of dimensionality, we recommend to first identify a 
subspace (e. g., by POD) with suitable basis matrix V corresponding to an orthogo- 
nal basis; then the training data can be projected orthogonally to the reduced space 
(u;, V’ Kx(u;)) and the mapping from input parameters to reduced state vectors can be 
learned. Such an approach, however, does not involve any knowledge about the sys- 
tem structure. Alternative approaches would consist of assuming a certain (paramet- 
ric) system structure and inferring the missing parameters purely from the observed 
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data. These approaches in MOR are denoted to be “data-driven.” In control theory 
such problems already have a long tradition in the area of system identification. As 
this, however, is not so much related to MOR software and interplay between full and 
reduced solvers, we do not comment on such options. 


13.2.2 Model 2: time-dependent ODE systems 
As second model we assume to have a time-dependent nonlinear ODE system, 
x(t) = f(x(t),u(f)), x(0) = Xo, (13.8) 


on a time interval [0, T] with final time T € RU {oo}, unknown state x : [0, T] > R”, 
input u : [0, T] > R™, and system nonlinearity f : R” x R™ — R”. Continuity of 
the right-hand side then implies existence of a solution. Additionally, there may be an 
output y : [0, T] — R”, which, for simplicity, we assume to be linear in the state 


y(t) = Cx(t) 


for C € R™*". We now focus on local evaluations of the system nonlinearity, as this 
will be the key for full online-efficient reduction. For a given set of few indices I = 
{i,,...iy} c {L...n} with typically M < n (e. g., DEIM “magic indices,” Chapter 5 in 
Volume 2 of Model order reduction), we define 


fr = foo- fa) (13.9) 


as local evaluation of f = (f;)};. We now require the following, which are typically 

nontrivial for given off-the-shelf simulation packages: 

G) The local evaluation f; can be computed without assembling the full nonlinear- 
ity f.! 

Gi) The evaluation of those components can be performed rapidly based on only a 
subset of the state vector x = (x;){; in the following sense: There exists an in- 
put index subset J := {i ... iş}, typically with n > M > M, that defines x; := 
OG, sx G) € RM as the restriction of x to the indices I. Then we assume that 
there exist functions fs, RMR > R,m = 1,...M, such that fin (x,u) = fs, (xu). 
This means that overall there is a function f; := (f>... f)" : RY x R™ > RM 
satisfying 


f(x; u) = f,(x, u) (13.10) 


1 Note that in the DEIM literature frequently the sampling matrix P := [e;,,...e;,] € R®™ is defined, 
where e; € R” denote the standard Euclidean basis vectors. Then we verify that fy = P’f, but the 
latter equation may give the wrong understanding that the sampling matrix multiplication is a com- 
putational recipe, which it is not, as this is of complexity n. Thus, we refrain from adopting this DEIM 
notation here and use the definition (13.9). 
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for any x and u. We emphasize that M, M, n,, typically are small, thus this function 
f; can be evaluated in complexity independent of n, and hence is online-efficient. 
If the system corresponds to a discretized PDE these properties are useful (and re- 
alistic): Property (i) corresponds to a local evaluation of the system nonlinearity 
in a few grid points. Property (ii) means that such a pointwise evaluation, e. g., 
of a finite difference pencil, only requires the knowledge of the state in a local 
neighborhood around the evaluation grid points, also understandable as a sub- 
mesh. This property is typical for discretizations of differential operators using 
basis functions of local support such as finite difference, finite element, finite vol- 
ume, or discontinuous Galerkin bases. If the model has some physical meaning, 
there may exist a weight matrix K € R™” that represents a physically relevant 
inner product and norm ||x||x := Vx! Kx. 


13.2.2.1 White-box reduction scenario 


Note that for linear time-invariant (LTI) systems, i.e., f(x,u) = Ax + Bu, there are 
plenty of reduction strategies, e. g., approximating the state trajectories for special in- 
put signals (snapshot-based such as POD, POD-greedy using the proper inner product 
given by K) or approximating the input-output behavior in some sense (e. g., optimal 
H>-approximation, moment-matching, Hankel norm approximation, etc. [2]). Again 
for simplicity, we assume to have some basis V € R™®” with r « n of orthogonal 
columns obtained by orthogonalizing any basis matrix of any of the mentioned pro- 
cedures. 

Then, a straightforward Galerkin projection of the system results in an approxi- 
mation x := Vx, with x, : [0, T] — R” being the solution of 


x,(t) = V'F(Vx,(0,u(0), x,(0) = Xo (13.11) 


with initial data projection x, := V' x and potential output approximation y(t) = 
Cx(t) = CVx,(t). Even though this is a low-dimensional model, it is not online- 
efficient, as (a) the reconstruction Vx,, (b) the nonlinearity evaluation, and (c) the 
projection V’f are operations of the original complexity n. This is a well-known com- 
putational bottleneck of reduction methods such as POD. In fact, the integration 
of (13.11) may even be more expensive than integrating the original system (13.8), ren- 
dering the reduced model practically useless. To decrease this computational com- 
plexity, sampling-based approximations of the nonlinearity are applied, for example 
by gappy POD [11] or POD-DEIM [8]. Here, a further approximation stage is employed: 
An additional basis matrix U is assumed that approximates f well by f := Uc. The 
coefficient vector c is computed by a linear transformation of a local nonlinearity 
evaluation c := Mf, as introduced in (13.9). This can both be an interpolation-type 
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approximation, e. g., DEIM,” but as well a least-squares approximation by oversam- 
pling of f. In both cases, U must approximate the range of f well, such as obtained 
by an additional POD of snapshots of the nonlinearity [8], or a greedy basis based 
on f snapshots (cf. the early references [16, 15]). If we replace f by f in (13.11) and use 
v'f = V'UMf, = Wf, with W := V'UM «€ R” and (Vx,); = V;x, with V; € R” con- 
taining the rows of V corresponding to the indices in I, we obtain a “hyperreduced” 
model as alternative to (13.11): 


x, (t) = V’F(Vx,(t), u(t) = WE,(Vx,(¢), u(t) = WE,(V;x, (0), u(t), (13.12) 


assigned with the identical initial conditions and a potential output as (13.11). 

We give some intuition about the operations involved in solving such a system: 
In the offline phase the initial state is projected, i. e., X, o is computed, and the small 
matrix W is assembled. Also we identify the appearance of a restriction of an RB V}. 
This means, for example in the case of a finite difference discretization, that the RB 
needs to be stored nodewise on “all” finite difference nodes in the stencils around 
the sampling points J. This matrix is of small size. Now, when turning to the online 
phase, i. e., integration of (13.12), we clearly see the requirement of repeated evaluation 
of the local nonlinearity evaluation function f; that we assumed to be available by 
the high-fidelity model and should be rapidly evaluated in a complexity independent 
of n. Also, instead of reconstructing the full state approximation Vx, as required in 
the model (13.11), the hyperreduced model only requires a local reconstruction of the 
approximated state V;x, which corresponds to the values of the reduced solution ona 
subgrid defined by the indices I. This local reconstruction of the state then is sufficient 
to exactly evaluate the system nonlinearity in the local sampling points required for 
the evolution of the reduced model. 

This concludes the “clean” white-box reduction scenario. We are not aware of im- 
plementation of all of the full model requirements in commercial packages to real- 
ize this. In particular the rapid local evaluation of the system nonlinearity is an ex- 
tremely intrusive — and mostly also code-intrusive — requirement of making complex 
discretization packages suitable for online-efficient model reduction. Some publica- 
tions that presented this ideal reduction for nonlinear problems include [16, 10, 8]. 


13.2.2.2 Gray-box reduction scenario 


Now again, as white-box scenarios are rarely implemented in usual high-fidelity sim- 
ulation packages, we again must refrain to gray-box scenarios. 


2 Note that in the case of the DEIM, the matrix U is of sizen x M and M = (PTU). Then obviously the 
interpolation property is verified: 


P’f = P'UMf, = P’U(P"U) fy =f, =P'f. 
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Missing inner product weighting matrix K 

If no weighting matrix K is available by the full model, the same workarounds as in 
Section 13.2.1.2 apply, i. e., working with the Euclidean inner product or reconstructing 
K by potential knowledge about the underlying PDE discretization. 


Missing local evaluation routine f, 

In the context of nonlinear problems, the main problem is a potential missing local 
evaluation routine. If no access to the full model apart from full samples of f are possi- 
ble, then one could sacrifice online efficiency for accuracy and just accept the (ineffi- 
cient) nonhyperreduced model (13.11). As mentioned above, conceptually a dimension 
reduction will be obtained by this, but no or limited computational gain is to be ex- 
pected. 

If aiming at online efficiency, one can add yet another approximation stage, e. g., 
realized by the Kernel-DEIM approach [39, p. 83ff], [22]: The idea is to provide kernel- 
based approximants for the local function evaluations: f; (Vx,) = fi, Vi%)s where 
Ê, is for instance a kernel interpolant or a more general approximant (Chapter 9 in 
Volume 1 of Model order reduction). This can be constructed purely based on state and 
nonlinearity snapshots. 


Missing local evaluation set / 

If the discretization/interpretation of the full model is unknown, it may be a priori un- 
clear which state entries influence the required sampling entries of the nonlinearity, 
i. e., the index set J may be unknown. A general approach for this is suggested: If the 
full nonlinearity of the system can be sampled, snapshot-based POD can be computed 
and a DEIM can be executed resulting in a possible choice of sampling points J. Now, 
the set of indices I that the magic index evaluations depend on must be generated. One 
possibility for this is analysis of the Jacobian of f (which clearly requires this as addi- 
tional functionality from the full solver): The nonzero entries in row i,, of the Jacobian 
indicate those entries of the input state vector that induce changes in the value of f. 
Thus, analysis of the sparsity pattern of Vf is sufficient for identification of the set I. 


Alternatives 

Now clearly, also other gray-box approaches can be followed if one refrains from the 
above Galerkin projection structure. For instance by working with state and velocity 
snapshots of a linear system, dynamic mode decomposition (DMD) [25] allows to infer 
an approximate linear model. Note that velocity, i. e., nonlinearity snapshots, require 
a sort of intrusiveness as this is more than just state observations, and therefore not 
black-box. If a system can be observed in terms of input and output observations in 
frequency space, then the Loewner framework [20] enables to efficiently find an ap- 
proximate LTI realization of a system with the observed behavior. For parametric un- 
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steady problems, e. g., LTI systems, the parametric matrix interpolation idea can also 
be applied (e. g., [13]). 


13.2.2.3 Black-box reduction scenario 


The mentioned gray-box approaches still assumed to have access to the systems non- 
linearity (or frequency response measurements). Now, what can be done if the only 
way of accessing the system consists of sampling state trajectories and outputs? First, 
a direct mapping of input state to output quantities can be approximated with ma- 
chine learning techniques. However, such techniques do not involve any model knowl- 
edge nor model assumptions. If one assumes a certain model structure, i.e., linear or 
polynomial nonlinearity for f, then the system coefficients of a reduced model can be 
inferred from observed data [31]. In control theory this field is long known as system 
identification. 

If one is interested in approximating x instead of an output, the prediction of 
a high-dimensional target quantity may suffer from the curse of dimensionality. So 
blended approaches combining model reduction and machine learning exist. For ex- 
ample [38] suggest a nonintrusive reduced-order model (ROM) approach for nonlinear 
problems combining POD and neural networks: By sampling state trajectories, a suit- 
able approximating space/basis V can be computed by POD. Then each of the state 
snapshots can be mapped to the POD space, obtaining POD coefficients. Then a neural 
network can be learned to map time to the POD coefficients, which thus directly pre- 
dicts an approximation of the state even without integration of a dynamical system. 


13.2.3 Coupling techniques 


In the above we have seen that online efficiency in MOR is based on a tight interplay of 
full-order solvers and MOR implementation. This requires information exchange be- 
tween those levels in suitable ways. Therefore, we now discuss the main two different 
coupling options for MOR software. As additional reference on the coupling aspect, 
we want to point to a presentation that also discusses various approaches for MOR 
interfaces,’ with a focus on motivating the package pyMOR as mentioned in the next 
section. 


Write high-dimensional data to disk 

The principle is the following: The full solver is agnoscent of the reduction scheme and 
reduction package but provides required high-dimensional data by file output, e. g., 
finite element method system matrices, or system matrix components (in the case of 


3 https://www.stephanrave.de/talks/cse_2015.pdf 


13 MOR software —— 445 


parameter separability) L? or H! scalar product weight matrices, grid description for 
visualization, etc. Also, the user or the full solver then must provide or export the co- 
efficient functions of the separable parameter decompositions as function strings, etc. 

The reduction scheme or MOR package then needs to import these data, project 
the high-dimensional objects to low-dimensional quantities, assemble the reduced 
system, and rapidly solve the reduced system. Potentially, the MOR code exports then 
high-dimensional state approximations for visualization or postprocessing by the full 
solver package. We refer again to Figure 13.1 for illustration of these exchange steps. 

This approach by file exchange is applicable for linear problems or even problems 
with low polynomial structure or suitably Taylor-approximated systems. 

There are several advantages or beneficial options resulting from this decomposi- 
tion: One can realize a straightforward coupling of different programming platforms 
for the full and MOR solvers by only agreeing on a joint file format. By this, the MOR 
code can be written in language of choice and can be agnoscent of the programming 
language of the full system. Also, full solvers can be easily exchanged. One can realize 
a full decoupling of high-dimensional offline and online operations. 

Some drawbacks and limitations of this approach are apparent: The reduced 
model simulation might be slow due to the disk access bottleneck. Further, the MOR 
code needs to process high-dimensional data. File export routines for the full solver 
must be implemented, which might be nontrivial (e. g., mass matrices in matrix free 
full solvers). This file exchange does not work for nonlinear problems or problems with 
complex parametric system components unless gray-box strategies, i.e., “hacks” or 
approximations of the preceding sections, are applied. Also, this full decomposition 
of offline and online phases prevents modern adaptive simulation schemes, where 
offline and online phases are blended (e. g., optimization with reduced model adap- 
tation, local MOR with adaptive basis enrichment, etc.). It may be that iteration of full 
solver steps, reduced system creation, and solve are impossible or difficult. 

Alternatively, the file exchange can also be based only on reduced quantities, if the 
full solver package is extended by MOR basis generation and projection functionality. 
Then the MOR package only needs to read reduced quantities, solve reduced systems, 
and communicate reduced coefficient vectors to the full solver package that is respon- 
sible for visualization/ postprocessing. 


Communication of full and reduced solver by function interfaces 
The principle here is that the full solver is extended in order to provide function access 
to high-dimensional quantities used for the reduction, e. g., inner product matrices, 
system nonlinearities, etc. An improved approach even consists in avoiding access to 
matrices, but only providing access to matrix-vector multiplication for inner-product 
matrices or system matrices. 

This offers some clear advantages: It is potentially very fast if optimal and min- 
imally invasive connection to the full solver is realized. Nonlinear problems can be 
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treated by giving the MOR package access to the exact system nonlinearity. The pro- 
visioning of matrix-vector multiplication routines allows matrix-free operations, and 
hence MOR for discretizations that do not rely on matrix representations (e. g., finite 
volume or finite difference discretizations). A potential tight and repeated interaction 
between full and reduced solvers is possible in modern simulation scenarios with 
adaptive ROMs such as in optimization or local MOR. If those function interfaces are 
designed in a minimal and generic way, very general model reduction is possible, as 
various full solvers can implement those interfaces and various MOR packages can 
use those different full solver interface implementations. If the interface classes make 
use of (references to) high-dimensional objects stored in the full solver’s memory, no 
communication of high-dimensional data is required, and the MOR code can be very 
fast and memory-efficient. 

Again, also clear disadvantages appear: This approach is more complex as cou- 
pling via online bindings, shared libraries, mex-interfaces, or network communica- 
tion, etc., is required. The realization of the required internal access may be difficult or 
impossible in certain full solver packages (e. g., local system matrix access to matrix- 
free discretization operators). 


13.3 MOR software packages 


After these conceptual aspects of MOR software, we want to become very specific in 
this section and give an overview of existing MOR software packages. First we address 
commercial packages, which we understand as being developed by companies with 
commercial interest. Then, we give a brief survey of academic packages, developed at 
universities or public research institutions. By the overview of this section, users may 
be guided (to some extent) to select the suitable packages for their application case. 


13.3.1 Commercial packages with MOR functionality 


Many simulator packages have realized the need and usefulness of model reduction 

on top of the traditional simulation chain. The following is a short and certainly in- 

complete list of such commercial packages. We do not go into details but refer to the 

corresponding websites. Mostly — but with notable exception of Scilab — the following 

packages are closed-source and subject to license fees. 

ANSYS® (CADFem):* The package Model Reduction inside ANSYS provides reduction 
techniques for linear systems, in particular three-dimensional finite element from 
ANSYS Mechanical™, enabling piezoelectrical and thermomechanical models. 


4 https://www.cadfem.de/produkte/cadfem-ansys-extensions/model-reduction-inside-ansys.html 


13 MOR software —— 447 


The resulting low-dimensional matrices can be imported in ANSYS Simplorer®, 
MATLAB/Simulink, or other system simulation languages VHDL-AMS or SPICE. 
An early reference to the initial version “MOR for ANSYS” has been published [3]. 

Akselos Integra™(Akselos):° This package enables numerical simulation of large- 
scale mechanical assets. Among others, the underlying technique is the static 
condensation RB element method [19]. Being based on reduction of components, 
this technique is very suitable for technical component-based structures. Linear 
systems can be treated, as well as systems with local nonlinear subsystems. 

CST MICROWAVE STUDIO® (CST):° In order to obtain network models, that are com- 
patible with the circuit simulator SPICE and have the same port behavior as three- 
dimensional structures, suitable MOR techniques can be applied. In particular 
stability and passivity preservation is obtained by the reduction techniques. 

MATLAB®” (The Mathworks, Inc.):8 In its control system toolbox various control- 
theoretic reduction techniques are provided, in particular pole zero simplifica- 
tion, mode selection or balanced truncation (BT) (Chapter 2 in Volume 1 of Model 
order reduction). These methods are available in the Model Reducer App, which 
can be interactively used. 

MOR toolbox (MOR DIGITAL SYSTEMS):? The MOR toolbox is a MATLAB-based tool- 
box gathering algorithms for (i) reduction of large-scale linear dynamical models 
and (ii) creation of linear dynamical models from input-output frequency data. 
The algorithms gathered in the MOR toolbox generate a linear state-space model 
whose input-output behavior is close to the initial model. Some methods take ad- 
vantage of the sparse nature of the models and can therefore be applied to very 
large-scale models with several thousands of states. Such models arise very often 
in physics, biology, etc. The MOR toolbox is free for academic use, and licenses 
are available for commercial purposes. 

SLICOT (Niconet ev,):° This collection of MATLAB toolboxes also contains a SLI- 
COT Model and Controller Reduction Toolbox, providing algorithms for many 
control-theoretic reduction methods, e. g., BT, singular perturbation approxima- 
tion, frequency-weighted balancing, Hankel norm approximation, or co-prime 
factorization. 

SciMOR (ESI Group, Scilab Enterprises SAS):!' This model reduction toolbox has re- 
cently been developed and provides modern techniques for reduction of paramet- 


5 https://akselos.com 

6 https://perso.telecom-paristech.fr/begaud/intra/MWS_Getting_Started.pdf 

7 We acknowledge MATLAB to be a registered trademark throughout this chapter, but for readability 
refrain from indicating this at various further occasions. 

8 https://de.mathworks.com/help/control/ug/about-model-order-reduction.html 

9 http://mordigitalsystems.fr/en/ 

10 http://slicot.org/matlab-toolboxes/model-reduction 

11 https://scilab.io/scilab-model-reduction-toolbox 
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ric PDEs. In particular, POD in combination with parameter interpolation is im- 
plemented. This toolbox is part of the Scilab open-source project with the goal of 
democratizing computational science and is free of charge. 


There are some common limitations with most of those MOR implementations: Apart 
from very recent developments, those commercial packages typically do not provide 
the most efficient latest MOR methods. This in particular holds true for packages that 
are already more than 10 years old, as the last decade has shown tremendous improve- 
ment in MOR technology. In these packages, in particular the full models are typically 
not fully accessible. As discussed in the previous sections, this prevents white-box 
implementation of modern reduction techniques, which mostly require a high level of 
intrusiveness. 


13.3.2 Academic MOR software packages 


Next, we want to give short descriptions of existing MOR software packages developed 
by groups from academia. The packages are free of charge, but may be subject to some 
licensing options. The packages are mostly open-source, except two of them, which 
are not publicly available for download. Typically, those packages are developed at 
universities or public research institutions. 

The software packages differ in various aspects. In addition to the open-source 
policy, the main aspect is the programming language. Differences can also be ob- 
served with respect to the ease of installation, whether paper references are given, 
and whether documentation, benchmark models, or tutorial examples are provided. 
Some packages are under active development by large development teams, while 
some are single-programmer projects that resulted from PhD theses and are “frozen” 
or only updated in a minimal fashion. Large differences exist in the provisioning of 
full-scale solvers within the package and the extent of the coupling to external high- 
fidelity models/solvers. The purpose of those packages can be either fundamental 
research, teaching, or even industrial application. The physical application fields 
also vary widely: Some packages only support one application domain (e. g., elec- 
tronics or mechanical systems) while others do not restrict the type of applications, 
but allow all kinds of physical domains, including electronics, fluid dynamics, biol- 
ogy, chemistry, finance, etc. Most discriminating are the types of systems that can be 
reduced such as LTI systems, nonlinear ODE systems, parametric problems, elliptic 
PDEs, parabolic PDEs, hyperbolic PDEs, first-order systems, second-order systems, 
differential algebraic equations (DAEs), and parametric, dense, or sparse systems. Fi- 
nally, the implemented reduction techniques are very different in the packages, e. g., 
BT, moment-matching (Chapter 3 in Volume 1 of Model order reduction), RB methods, 
POD, Hankel norm approximation, optimal H,-norm reduction, etc. 

In Table 13.1 we give some metadata for several packages from academic devel- 
opment teams that are available at the time of finalization of this overview, i.e., June 
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Table 13.1: Metadata of academic MOR software packages (as of Aug. 2019). 


Acronym Ref. Language License type Latest version 

DPA [33] MATLAB /Octave - Nov. 2015 

emgr [18] MATLAB /Octave BSD-2-Clause 5.8, May. 2020 
ITHACA [36] C++ LGPL 3.0 / MIT License Mar. 2020 

KerMor [39] MATLAB GNU GPL & BSD 0.9, Aug. 2015 
M.E.S.S. [34] MATLAB/Octave, C GNU GPL 2 2.0.1, Feb. 2020 
MOREMBS [12] MATLAB, C++ - on request 
MORLAB [5] MATLAB GNU Affero GPL 3 5.0, Dec. 2019 
MORPACK [26] MATLAB - on request 

pyMOR [27] Python BSD-2-Clause 2019.2, Dec. 2019 
RBmatlab [14] MATLAB - 1.16.09, Sept. 2016 
RBniCS [17] Python GNU lesser GPL 3 v0.1.0, Jun. 2019 
SparseRC [21] MATLAB - Nov. 2011 
sssMOR [6] MATLAB BSD-2-Clause v2.00, Sept. 2017 


2020. We do not express any preference by the order of the packages but present them 
in alphabetic order. For each of the packages we give a reference that either is specif- 
ically devoted to that software package or is a reference using that package. We spec- 
ify the corresponding (main) programming languages, not excluding that some of the 
packages provide bindings or some optimized routines for some other language. Most 
of the packages are MATLAB-based, some using as little MATLAB-specific function- 
ality that they are also executable from Octave. The packages devoted to MATLAB or 
Python do mainly not have restrictions on the operating system, as long as MATLAB 
or Python is installed in corresponding versions. Only the package versions providing 
C/C++ versions typically are limited to either Windows or Linux operating systems. 
About half of the packages specify some open-source GNU- or BSD-type license. The 
column “latest version” lists version numbers and release dates if provided by the sup- 
porting websites. Some packages do not provide access by download but only on re- 
quest, in particular packages that are used for simulation of multibody systems (Chap- 
ter 2 in this volume). The recent release dates indicate that most of the packages are 
under active development and we recommend to consult the corresponding webpages 
and github sites for most recent information. Especially the packages maintained at 
github frequently have most recent commits that yield functional versions more re- 
cent than the latest release tags given in the table. In particular we want to emphasize 
that the following detailed descriptions may relate to more recent git-commits than 
the versions mentioned in the table. In the context of MOR software, we want also to 
refer to the excellent software list at the MOR Wiki.” Note that, by nature, our package 
list has a considerable overlap to that online reference list. 


12 http://www.modelreduction.org 
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We now give some more details on the packages, including nonabbreviated name, 
open-source policy, online availability, license type, download (or project) URL, pro- 
gramming language, installation procedure, documentation, main reduction meth- 
ods, application fields, and further individual comments. 

DPA:” A collection of MATLAB algorithms related to versions of the Dominant Pole 
Algorithm [33] are provided for download as source code without license restric- 
tions. No installation steps are required, and operation is performed via simple 
execution of the *.m files. Documentation is available as comments in the pro- 
gram files. Reduction by those methods is related to modal reduction with connec- 
tions to moment-matching (Chapter 3 in Volume 1 of Model order reduction). The 
algorithms allow reduction of first- and second-order single-input, single-output 
systems and first-order multiple-input, multiple-output systems. The motivating 
field of application is power system electronics, but the code can be used for re- 
duction and modal analysis of dynamical systems from other domains as well, 
including electronics (RLC parasitics), acoustics, and mechanics. Quite a number 
of test systems and (large-scale) system matrices are provided. 

emgr:’ The MATLAB/Octave package provides an Empirical Gramian Framework [18] 
for model reduction of nonlinear input-output systems. The program files are 
available for download under the open-source BSD-2-clause license. No installa- 
tion steps are required, as the single MATLAB file can directly be executed without 
dependency on further packages. The empirical Gramians basically extend the 
concept of system Gramians for first-order LTI systems (Chapter 2 in Volume 1 of 
Model order reduction) to nonlinear systems. Overall, these reduction techniques 
can be related to BT. For parametric problems with high-dimensional parame- 
ter spaces, empirical Gramians also allow combined reduction of parameter and 
system order. Due to its generality, there is no restriction to the field of applica- 
tion: Models from neural science, mechanical systems, electrical networks, or dis- 
cretized PDEs are contained as benchmark systems on the website. Extensive doc- 
umentation of the programs is also provided online. 

ITHACA:” This C++ package on In real Time Highly Advanced Computational Ap- 
plications comes in several versions: ITHACA-FV with finite volume full-order 
solver https://mathlab.sissa.it/ITHACA-FV [36, 37] based on OpenFOAM, ITHACA- 
SEM with spectral element detailed solver https://mathlab.sissa.it/ITHACA-SEM 
based on Nektar++ and ITHACA-DG https://mathlab.sissa.it/ITHACA-DG based 
on discontinuous Galerkin full-order solver based on HopeFOAM. All versions 
are open-source, the former under a LGPL license, the latter two under an MIT 
License. The packages’ code and documentation are available on corresponding 


13 https://sites.google.com/site/rommes/software 
14 https://gramian.de 
15 http://mathlab.sissa.it/ 
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github pages. The packages ITHACA-SEM and ITHACA-DG are at an early de- 
velopment stage, thus we focus on the more mature ITHACA-FV package in the 
following: The installation of ITHACA-FV requires an existing implementation of 
OpenFOAM 5.0, 6.0, or 1812, and then the cloning and compilation of the package 
are realized with few commands. The package provides several well-documented 
tutorial examples. The documentation is extracted from the code by doxygen. 
The package provides the implementation of several reduced-order modeling 
techniques for parameterized problems. In particular, the thermal block, steady 
and unsteady Navier-Stokes, additionally coupling with an energy equation, and 
the Boussinesq equation are contained. As reduction techniques, POD, nonintru- 
sive POD-interpolation, DEIM (Chapter 5 in Volume 2 of Model order reduction), 
and DMD (Chapter 7 in Volume 2 of Model order reduction) are provided. 

KerMor:'© The MATLAB package provides Kernel and MOR methods for surrogate mod- 
eling of nonlinear systems [39]. It is open-source partly under the GNU GPL 3 and 
BSD license. The source files are maintained and are accessible freely at a cor- 
responding github repository. The package is using sophisticated object-oriented 
features of MATLAB, and hence is not suitable to be used under Octave. Program 
documentation is provided online but can as well be generated offline by the 
mtoc++ and doxygen documentation tools. After download or cloning of the pack- 
age, some installation steps are required for compiling suitable mex functions and 
setting environment variables. Then a single startup file needs to be executed in 
MATLAB in order to use the package. The considered model classes are simple 
IO function maps, LTI and parametric nonlinear systems. As surrogate model- 
ing techniques, mainly projection-based methods (POD-DEIM) (Chapter 5 in Vol- 
ume 2 of Model order reduction) and kernel methods (VKOGA, SVR) (Chapter 9 in 
Volume 1 of Model order reduction) are provided. The field of application is not 
limited, demo examples contain electric circuit as well as system-biological (pro- 
grammed cell death) models, discretized PDEs (Burgers), or biomechanics (non- 
linear elasticity for muscle models). 

M.E.S.S.:"” The Matrix Equation Sparse Solver library [34] provides algorithms for 
approximate matrix equation solving such as large-scale Lyapunov or (differen- 
tial) Riccati equations. Since the solution of such matrix equations represents the 
core of many algorithms in model reduction and control, this package is essen- 
tial for reduction of large-scale problems. The package consists of a version for 
MATLAB/Octave and a version for C; additionally it provides Python bindings. 
The code is accessible via a public git repository and covered by a GNU GPL 2 
license with some exceptions. The MOR functions comprise BT and the iterative 
rational Krylov algorithm (IRKA) (Chapter 3 in Volume 1 of Model order reduction) 


16 https://www.morepas.org/software/kermor/index.html 
17 http://www.mpi-magdeburg.mpg.de/projects/mess 
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for first-order LTI systems. Documentation is provided within the MATLAB source 
files. Installation is straightforward by simple unpacking and a startup script 
for initialization at each MATLAB session. Due to its generality, model examples 
both contain discretized transport PDEs (heat equation, advection-diffusion) and 
mass-spring-damper systems for structure-preserving second-order techniques. 
Further demonstration examples explain the extension to structured DAE systems 
that allow for implicit index reduction. 

MOREMBS:’® This package on Model order reduction for elastic multibody systems 
[12] consists of both a MATLAB version (MatMorembs) and a C++ version 
(Morembs++). The package is not available for download, but can be provided on 
request for academic use. The supported reduction techniques for second-order 
systems comprise modal techniques such as the Craig—-Bampton method for com- 
ponent mode synthesis (CMS), as well as Krylov subspace techniques (Chapter 3 
in Volume 1 of Model order reduction) or SVD/Gramian-based techniques (Chap- 
ter 2 in Volume 1 of Model order reduction). The field of application is clearly 
focused on elastic mechanical systems. The strength of the package lies in a mul- 
titude of coupling options, in particular importing system matrices from various 
commercial finite element programs (ABAQUS, ANSYS, PERMAS, Nastran) and ex- 
porting reduced system descriptions to multibody simulator programs (MATLAB 
Simulink, SIMPACK, Neweul-M’, Adams, LMS). 

MORLAB:” This Model Order Reduction LABoratory package [5] is aiming for spectral- 

projection-based model reduction of dynamical systems. It is freely available for 
download as a MATLAB toolbox, Octave package, or zip archive. The package is 
subject to the GNU Affero General Public License 3. Installation is simple by exe- 
cuting a startup file for adding paths or by using the automatic mechanisms for 
MATLAB toolboxes and Octave packages. 
There is an extensive HTML documentation included in the package. The pack- 
age aims at dense first-order LTI, descriptor, or second-order systems. The spec- 
trum of methods is based on the solution of matrix equations [29], in particular 
modal truncation, BT with many variants (frequency-limited BT, time-limited 
BT, bounded-real BT, positive-real BT, balanced stochastic truncation, linear 
quadratic-Gaussian BT, and H,,-BT), as well as a variant of the Hankel norm 
approximation. Due to its generality of systems, the package does not focus on 
special application fields. 

MORPACK:”° This Model Order Reduction PACKage is a MATLAB library for reduc- 
ing elastic multibody systems [26, 24]. The package is not available for download 
but test versions for academic purposes can be provided on request. The appli- 


18 http://www.itm.uni-stuttgart.de/research/morembs/MOREMBS _en.php 
19 http://www.mpi-magdeburg.mpg.de/projects/morlab 
20 https://tu-dresden.de/ing/maschinenwesen/ifkm/dmt/forschung/projekte/morpack 
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cation field is limited to second-order mechanical systems for elastic multibody 
dynamics. The package acts as a general interface between finite element soft- 
ware (ANSYS, ABAQUS, NASTRAN, LS-DYNA) and the multibody simulators (SIM- 
PACK, ANSYS, EMBS-Matlab). Therefore, validation of the reduction and choice 
of the master degrees of freedom have a high priority. Over 60 correlation criteria 
are available to compare reduced-order models against the original model as well 
as measurement data. Likewise, there are many methods to select optimal master 
degrees of freedom. The reduction methods that are available comprise Guyan, 
CMS, Krylov subspace methods, (second-order) BT, and minimal model genera- 
tion by mode truncation. Also multistep reduction processes can be performed. 
The package is steered by a graphical user interface. Several benchmark models 
are provided ranging from 100,000 to 1,200,000 degrees of freedom. 

pyMOR:” The Model Order Reduction with Python package is an open-source library 
under active development [27]. It is accessible by a repository at github under 
a (modified) BSD-2-Clause license. Installation basically works via pip, and de- 
tailed installation instructions are given on the website. Extensive program doc- 
umentation is provided online and within the program source code. The pack- 
age aims at covering all types of reduction techniques, ranging from RB methods 
for parameterized PDEs up to MOR for control systems, e. g., BT or Krylov sub- 
space methods. Also general nonlinearities can be treated by empirical interpo- 
lation (Chapters 1 and 5 in Volume 2 of Model order reduction). Due to its gener- 
ality, no application fields are excluded. By using abstract interfaces, coupling of 
external high-fidelity solvers is possible and several of such dockers exist, e. g., 
for Dune, FEniCS, deal.II, or NGSolve. Also, some finite element and finite vol- 
ume discretizations are included based on NumPy/SciPy. Two jupyter notebooks 
are provided for interactive exploration of corresponding models (heat equation, 
spring) and reduction techniques. Many demo applications are contained within 
the package such as PDE-based models (Burgers equation or elliptic and parabolic 
equations). 

RBmatlab:” The Reduced Basis Matlab package is an open-source library for numer- 
ical approximation of parameterized problems. The code is publicly available for 
download via the Model Reduction of Parametrized Systems (MoRePaS) website 
without license restrictions. The master branch is maintained as a git repository, 
for which access can be granted on request. Documentation is provided online 
and within the MATLAB function headers. This documentation can be generated 
offline by the mtoc++ and doxygen tools. Installation is simple by unzipping, set- 
ting two environment variables, and optionally extending the MATLAB startup 
script to get RBmatlab started automatically during the initialization of each 


21 https://github.com/pymor/pymor 
22 https://www.morepas.org/software/rbmatlab 
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MATLAB session. As system types parametric PDEs (elliptic, parabolic, hyper- 
bolic) mostly motivated by transport problems (heat equation, Burgers equa- 
tion, two-phase flow), mechanics (elasticity), or finance (variational inequalities) 
are supported, as well as parametric control systems, e. g., the chemical master 
equation. Snapshot-based reduction methods (POD, greedy, POD-greedy) are im- 
plemented, including certification by error estimators. Coupling to the scientific 
computing packages Dune, Alberta, and COMSOL is realized. Additionally, the 
package contains PDE discretization techniques (finite element method, finite 
volume method, discontinuous Galerkin) to be used as high-fidelity solvers for 
reduction procedures. Use of those reduced models in parameter optimization 
and feedback control are some of the implemented multiquery settings. Many 
demos are implemented and can be interactively accessed for getting insight into 
the functionality of the package. A pedagogical model of the well-known thermal 
block model is provided as tutorial example [14]. 

RBniCS:” This package on Reduced Order Modelling in FEniCS is understood to be ac- 
companying the book [17]. Installation prerequisites are the availability of FEniCS 
(with PETSc, SLEPc, petsc4py, and slepc4py), numpy, and scipy. The remaining 
installation of RBniCS is then easily done by cloning the git repository and re- 
questing python3 to install the package. As model classes the package comprises 
parametric elliptic and parabolic problems. Both linear and nonlinear problems 
(using empirical interpolation) are considered. Advection-diffusion as well as 
Stokes and Navier-Stokes problems are readily available. The code uses clear 
naming, and is hence sufficiently comprehensive. A documentation can be gen- 
erated but is not available online. As basis generation procedures POD, greedy, 
and Gram-Schmidt algorithms are realized. The successive constraint method 
for rapid computation of stability factor lower bounds is implemented. Almost 
20 tutorials are available and suitable to be used in model reduction courses. A 
particular feature of this package is the ease of specifying new problems, i.e., 
not only changing coefficient functions but also specifying and changing the 
differential operators of the PDE by high-level FEniCS commands. 

SparseRC:™ SparseRC [21] is a collection of MATLAB routines which performs parti- 
tioning/reordering-based model reduction for RC netlists with nodes up to hun- 
dreds of thousands, and terminals up to tens of thousands. The motivating field 
of application is analog circuit design, where parasitic extraction of the physi- 
cal layout may result in large-scale networks of resistors (R) and capacitors (C). 
These networks can be modeled by dynamical systems and SparseRC can be used 
to reduce these systems, exploiting and preserving properties specific to such 
networks. 


23 https://mathlab.sissa.it/rbnics 
24 https://sites.google.com/site/rionutiu2/research/software 
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sssMOR:” The Sparse State-Space and Model Order Reduction Toolbox [6] and the 
extension psssMOR? for parametric problems are MATLAB libraries distributed 
under the BSD-2-Clause license. The code is accessible via a git repository at 
github. Installation is realized by unzipping the packages and executing a few 
installation commands as specified on the webpage. The MATLAB functions are 
very well documented, and hence accessible by the MATLAB help functionality. 
The particular motivation of those packages is extending the MATLAB-inherent 
state-space toolbox, which is restricted to dense matrices and thus only allows 
treatment of moderately sized problems. The sss toolbox provides this func- 
tionality using sparse matrices, and hence can treat considerably larger system 
orders. The sssMOR library then implements MOR techniques using those sparse 
system representations. The system types considered are LTI control systems. 
Basic reduction techniques implemented are modal truncation, BT, and rational 
Krylov subspace methods. At the same time, it provides the IRKA as well as some 
more recent algorithms such as the CUmulative REduction framework (CURE), 
the Stability-Preserving, Adaptive Rational Krylov algorithm (SPARK), and the 
confined IRKA (CIRKA). Some model samples from well-known benchmark col- 
lections (CD player, building, gyro) are provided. In principle, the application 
scope is not limited as long as the systems can be cast as (parametric) LTI con- 
trol systems. The toolboxes are currently being extended to cope with nonlinear 
sparse state-space systems, such as bilinear (bsssMOR) and quadratic-bilinear 
(qbsssMOR) models. Extensions for other system classes, e. g., port-Hamiltonian 
(spHMOR) and second-order (ssoMOR) systems, are naturally also conceivable 
under the same guiding principle. 


We do not claim completeness of this above package list, as many researchers have 
their private code collection, libraries, or repositories. But the list covers the main cur- 
rently available software packages that we are aware of. Several smaller packages ex- 
ist, such as pydmd?” on DMD and ezyrb? on POD with interpolation. Some academic 
PDE discretization packages also include MOR functionality, e. g., libmesh,”? which 
has RB capabilities via rb0Omit [23], or Feel++,*° which also provides MOR methods. 
Further packages exist which however are no longer under active development and not 
provided by download. Among those we want to mention dune-rb, whose capabilities 
and principles of coupling with RBmatlab have been explained in [9]. The thesis [1] 
describes the extension to localized model reduction approaches. Also the package 


25 https://www.rt.mw.tum.de/?sssmor 
26 https://www.rt.mw.tum.de/?psssmor 
27 https://mathlab.sissa.it/pydmd 

28 http://mathlab.sissa.it/ezyrb 

29 http://libmesh.github.io/ 

30 http://www.feelpp.org 
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rbMIT is a software package that provides the most elementary RB algorithms. This 
software package was awarded with the Springer Computational Science and Engi- 
neering Prize in 2009. The package is currently not available by a website, but can be 
accessed via the internet archive.” Similarly the package PABTEC on BT for electronics 
applications is listed at the swMATH portal’? but seems no longer to be publicly acces- 
sible. The academic MORE package” [32] is a precursor of the commercial MOR toolbox 
mentioned in the previous subsection. 

Disadvantages of academic packages (which more or less applies to the different 
packages mentioned above) must certainly also be stated. The level of documentation 
of those code packages may be incomplete, development of packages may be discon- 
tinued, and support can typically not be offered in an extensive manner or instanta- 
neously. So using these packages typically requires some self-study, reproducing of 
running models and reduction techniques, or even some reverse engineering, or trial 
and error with changing parameters. 


13.4 Conclusions and recommendations 


We argued that online-efficient and accurate MOR algorithms require access to data, 
functionality, or other internals of the full solver. In particular, in many cases, a spe- 
cial design, decomposition or structure of the full-order model needs to be established 
in order to optimally apply corresponding MOR techniques. Elementary routines from 
the full solver or high-fidelity simulation package that are required for white-box MOR 
may comprise the following: For snapshot-based MOR algorithms, e. g., POD, greedy, 
and POD-greedy, a triggering of a full-order simulation with specified input parame- 
ters and return of state snapshots must be available. Information about nodal inter- 
pretation of the state vector is required. In the simplest case of linear finite element or 
finite difference discretization this is equivalent to enumeration of the mesh nodes. In 
general, a routine for assembly or matrix-vector multiplication with the inner prod- 
uct (mass) matrix is very helpful. This enables computing L-norms, orthonormaliza- 
tion, and projections. Similarly, a subroutine for assembly or matrix-vector multipli- 
cation with the stiffness matrix, which enables computing Sobolev H!-semi-norms, is 
helpful. Export of other system matrices, e. g., Jacobian matrices of nonlinear terms, 
may be required. For parametric problems an export of parameter separable decom- 
positions is essential; this means access to (nonparametric) system matrix and vector 
components and parametric coefficient functions. For sampling-based methods for 


31 https://web.archive.org/web/20171110212818/http://augustine.mit.edu:80/methodology/ 
methodology_rbMIT_System.htm 

32 https://swmath.org/software/4061 

33 https://w3.onera.fr/more 
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nonlinear problems, local evaluation of system nonlinearities based on local recon- 
struction of the state vector needs to be provided. Optionally, visualization or post- 
processing of a given state vector by the full solver is useful. If those functionalities 
are not provided by the full solver, either a loss of online efficiency is unavoidable, 
or some gray-box or black-box workarounds or hacks might be possible as we have 
exemplified for general stationary and unsteady, linear, and nonlinear problems. 

A multitude of software packages from groups from academia have been devel- 
oped. Typically those packages not only provide the latest MOR technology, but also 
implementations of some full system solvers. Only by control of all implementational 
details of the full models, which are typically code-intrusive, optimal online efficiency 
can be realized. This is mostly not possible by full-order models from commercial 
packages. This full control of the high-fidelity models enables independence of sci- 
entists from external simulator packages, but at the same time is very time-intensive, 
distracting one from main disciplinary research. 

We want to close with some recommendations. Addressing commercial software 
developers, we think that simulation packages will not remain competitive if MOR 
technology is not included as essential enabler for modern higher-level simulation 
tasks such as uncertainty quantification, parametric studies, optimization, design, 
etc. Simulation software engineers should be aware that their solver is no longer the 
last element in the simulation analysis pipeline but those are embedded in more com- 
plex simulation tasks. Inclusion of MOR algorithms in simulator packages can thus 
be a competitive advantage. When planning to include MOR technology, one should 
be clearly aware that access to more internals than just the solution/state degree of 
freedom vector is required. Access to system matrices, components, local nonlinear- 
ity evaluations, local subgrid geometry, etc., can be useful or required for modern and 
efficient MOR algorithms as listed above. Creating suitable interfaces or other means 
of access to those internals will enable applying efficient MOR algorithms. Apart from 
exporting system parameters, matrices, or geometry information, also importing fa- 
cilities for effective bases, error estimation routines, etc., would be recommended for 
expanding the scope of applicability of commercial simulation packages. In particu- 
lar, by realizing error estimation techniques, a certification and therefore guarantee of 
reliability of methods/packages is obtained. The last decade has enabled hundreds of 
PhD students to specialize and graduate in the field of MOR. These would be excellent 
candidates for transfer of those technologies into business and industry. A further op- 
tion for realizing MOR algorithms is the foundation of public—private partnerships or 
joint programs, e. g., European Industrial Training Networks, etc. 

Some recommendations for academic MOR researchers might be to develop fur- 
ther “hacks” of industrial software, i. e., using solvers as black-box or gray-box leading 
to new algorithms and analysis. For reproducibility of results, we strongly encourage 
to provide open-source and online accessible program code. If possible, it is advisable 
to contribute to existing MOR libraries instead of reinventing the wheel by developing 
new packages from scratch. If it is required or desired to reinvent MOR code or solvers, 
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practitioners or simulation engineers should think of useful central interfaces, which 
will ease exchange and coupling of the code to other packages. This is particularly rel- 
evant for bridging different programming platforms such as C++, MATLAB, Python, 
Fortran, etc. 

A general wish for the development in computational sciences is the increased 
acknowledgment of and respect for the effort in development, maintenance, and 
documentation of software packages. Typically, applications and computational ap- 
proaches are so involved, that software packages cannot be set up from scratch by 
scientists within the typical scientific “lifetime,” e.g., a PhD thesis. In particular, 
existing open-source packages directly enable a high entrance level for further de- 
velopments. Software development thus is crucial and one core scientific service for 
the community. Scientists and students who develop, maintain, and provide well- 
designed software should more easily obtain scientific credits by accepting such 
results as major scientific results in theses as well as being able to publish journal 
articles on such software — in contrast to the current state of expecting such develop- 
ments as minor by-product of disciplinary scientific contributions. But also in funding 
agencies such major achievements or proposals on scientific software development 
should be accepted as foundation for disciplinary progress. Only slowly this aware- 
ness in the scientific communities is developing, e. g., reflected by the recent Software 
branch at the SIAM journal on Scientific Computing that now also enables to pub- 
lish journal articles on software. But certainly this general awareness can be further 
strengthened by each individual researcher. 
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-strong form 176 
-weak form 
—curl-curl 186 
- electrostatics (ES) 174 
- general electrodynamics (ED) 191 
- general electrodynamics (ED with ECE) 195 
— magnetic stationary (MG) 186, 189 
-well-posed 151, 176 
fourth-order accurate Runge-Kutta scheme 292 
full-order models (FOMs) 37, 204, 240, 252, 331 
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functional magnetic resonance imaging (fMRI) 
238 


Galerkin projection 4, 18, 38, 64, 174, 177, 243, 
258, 259, 280, 290, 324, 435, 441 

gappy POD 224, 441 

Gauss—Newton approximate tensor (GNAT) 
method 224, 225 

Gauss-—Ostrogradsky formula 174 

Gaussian process factor analysis 239 

Gaussian process regression 6 

generalized controllability Gramians 349, 358, 
370 

generalized Lyapunov inequality 133 

generalized observability Gramians 349, 358, 
370 

genetic programming 282, 299 

Gram-Schmidt orthogonalization 123, 436 

Gram-Schmidt orthonormalization 260 

Gramian matrix-based reduction 58 

Gramian-based method 34 

graph clustering 346, 353, 356, 366 

graph theory 348, 350 

Grassmann manifold 216 

Grassmann manifold interpolation 303, 304 

Grassmannian manifold 296 

greedy algorithm 245, 334, 436, 454 

Guyan reduction 39, 192 


Hamilton’s principle 90 

Hankel norm approximation 346 
Hardware-in-the-loop 402 

Helmhotz equation 79 

hidden Markov models (HMMs) 239 
hierarchical clustering algorithm 347, 360 
Hodgkin—Huxley (HH) cable equations 240 
Hodgkin—Huxley (HH) equations 238 

hp refinement 185 

hyperreduction 34, 55, 63, 66, 223 


implicit Newmark scheme 193 

incidence matrix 350 

inf-sup stability 259, 260 

infinite-inputs infinite-outputs (IIO) system 153 
Isomap 297 

Isomap79 239 

iterative rational Krylov algorithm (IRKA) 84, 451 


Johnson—Champoux-Allard equivalent fluid 
model 99 
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k-means clustering algorithm 262, 347 

Kalman decomposition 348 

Kalman filter 413 

Karhunen—Loéve decomposition 4 

Karush—Kuhn-Tucker (KKT) conditions 14 

kernel methods 451 

kernel-DEIM approach 443 

Kirchhoff—Love plate equation 78 

Kirchhoff—Love plate theory 78 

Kolmogorov N-width 214, 215, 230, 334 

Koopman analysis 281 

Kosambi-Karhunen-Loéve transform 288 

kriging 6, 11 

Kron reduction 347 

Krylov subspace method 33, 34, 38-40, 42, 65, 
84, 123, 126, 329, 346, 366, 452 

— matrix-free Krylov 95 

-rational Krylov subspace method 84, 85, 455 

Kullback—Leibler divergence rate 366 


Lagrange multipliers 187 

Lagrangian structure 90 

Lanczos algorithm 83, 123 

- explicitly restarted 132 

-implicitly restarted 132 

Laplacian matrix 351 

latent variables 239 

Lax—Milgram theorem 176, 177, 182 
least squares regression 6 
least-squares Petrov-Galerkin method 222 
localized DEIM 248 

locally linear embedding (LLE) 239, 297 
Loewner framework 94 

Lyapunov Riccati equations 451 


manifold learning 297 

manifold model 296 

matrix discrete empirical interpolation method 
(MDEIM) 260, 261, 438 

matrix pencil 133 

matrix rational approximation (MRA) 119 

mesh deformation technique 260 

mesh-based variational methods 260 

method of characteristics (MoC) 119 

minimum-residual method 221 

- minimum-residual collocation method 223 

missing parameter separable decomposition 
438 

missing point estimate method 229 


Mitchell-Schaeffer model 262 

modal approximation 84 

modal derivatives 63 

— static modal derivatives 63 

modaltruncation 84, 421 

model order reduction (MOR) 2, 34, 39, 77, 83, 
94, 111, 120, 125, 129, 159, 209, 220, 238, 
322, 432 

-a posteriori 146, 191, 193, 196 

-a priori 146, 196 

—code-intrusive 432 

-goal oriented MOR 213 

-on-the-fly 146, 185, 192, 196 

- projection-based 324 

model predictive control (MPC) 203, 405 

Model-in-the-loop 400 

modeling 

-analytical 146, 159, 173 

-computational 146, 147, 181, 184 

-conceptual 145 

-geometrical 145, 159 

—mathematical 145, see also formulation, 
well-posed, 147, 150, 152, 159 

-numerical see also numerical method, 146, 
174, 181, 185, 194 

-physical see also formulation, 145, 150, 159 

-reduction see also model order reduction 
(MOR), 146, 153, 156, 159, 164, 172, 174, 
180, 182, 185 

-verification and validation 146 

modified nodal analysis (MNA) 113, 131 

-time-domain 118 

moment-matching method 450 

- explicit moment-matching method 121 

-implicit moment-matching method 122 

MOR plug-in 392 

MOR-Wiki 34 

multi-input multi-output (MIMO) system 55, 123, 
153, 154, 244 

multidimensional scaling (MDS) 297 

multipolar electric circuit element (ECE) 153 

multipolar EM circuit element (EMCE) 155 

multivariate quadrature rule 325 


network systems 345, 347, 351, 367 

— consensus networks 351 

- linear network systems with diffusive 
couplings 347 


Networked nonlinear robustly synchronized 
Lur’e-type systems 347 

neural networks 6, 281, 444 

- long short-term memory (LSTM) 308 

Niconet Benchmark Collection 34 

nodalelimination 421 

nonlinear autoregressive model with exogenous 
inputs (NARMAX) 281 

nonlinear programming (NLP) 1 

nonlinear stochastic Krylov training sets 
(NSKTS) 65 

nonnegative DEIM (NNDEIM) 243 

numerical method 

- boundary elements 146 

-finite differences 146 

-finite element see also finite element method 

-finite integrals 146 


Oberwolfach Benchmark Collection 34 
Obreshkov-based methods 114 
Ockham’s razor 282 

offline phase 217, 225, 245, 331, 432, 435 
offline-online decomposition 59, 210, 217, 432 
online phase 217, 226, 245, 331, 432, 435 
operator 

-curl 148 

—curl-curl 164, 170 

-del 148 

-div 148 

-div-grad 162 

-grad 175 

-input-output transfer 152 

-kernel 150, 187 

— surface curl 148 

— surface div 148 

Optimal control 400 


Padé approximation 122 

Padé via Lanczos (PVL) method 123 

— matrix PVL(MPVL) algorithm 123 

— multiport counterpart (SyMPVL) algorithm 123 

— SyPVL algorithm 123 

parameter-domain decomposition 215 

parameterized coupled monodomain-ionic 
model 261 

parameters 

-distributed 153, 156, 158, 159, 169 

-frequency dependent 169 

-lumped 153, 161, 166, 169, 173 
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-transient 157 

parametric model order reduction (pMOR) 112, 
322, 331 

parametric regression 438 

parametric stationary variational problem 434 

parasitic 421 

Pareto analysis 282 

passive reduced-order interconnect 
macromodeling algorithm (PRIMA) 123, 
126, 131, 132 

Pearson’s p correlation coefficient 298 

perfectly matched layers 92 

Petrov—Galerkin approximation 346 

Petrov-Galerkin projection 38, 122, 219, 258, 
353 

POD-Gram method 56 

POD-greedy algorithm 215, 441 

polynomial chaos expansion 322, 327 

polynomial regression 298 

port-compression algorithms 129 

-ESVDMOR 129 

—RecMOR 129 

-SVDMOR 129 

port-Hamiltonian 38 

preconditioners 

- preconditioned Krylov solvers 83 

- shifted Laplace preconditioner 83 

- sparse lower-upper (LU) factorization 83 

preconditioning techniques 180 

pressure supremizing operator 260 

principal component analysis (PCA) 239, 246, 
288, 296 

-kernel PCA 297 

productizing 384 

proper orthogonal decomposition (POD) 3, 4, 
33, 58, 63, 100, 136, 211, 214, 238, 240, 
258, 262, 280, 287, 288, 296, 436, 454 

- balanced POD (BPOD) 212, 228 

-frequency-domain 212 

-time-domain 211 

prototyping 385 

psd-convex-concave decomposition 365 

pseudo-controllability Gramian 359, 362 

pseudo-Gramians 350 

pseudo-time continuation method 221 

pseudo-transient continuation (PTC) method 
207 


quadratic bilinear (QB) systems 324 
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radial basis functions (RBFs) 11 

randomized dependence coefficient (RDC) metric 
310 

reduced basis method 4, 252, 258, 333, 434 

- coarse algebraic least-squares 260 

-stochastic reduced basis method 335 

reduced-order bases (ROBs) 243 

reduced-order modeling 258, 290, 309, 347 

— neuroscience 

— energy-stable neuronal reduced-order 
modeling 242 
- morphologically accurate reduced-order 

modeling 240 

reduced-order models (ROMs) 37, 41, 202, 238, 
252, 280, 322, 346, 354 

reduced-space interpolation 216 

reduction scenario 

-black-box 433, 434, 439, 444 

- gray-box 433, 434, 436, 442 

-white-box 432-436, 441 

representation learning 297 

response surface methodology 6 

Ritz minimization 177 

Ritz—Galerkin projection 35 

Rogers—McCulloch model 262 

Runge-Kutta solver 60 


sampling schemes 

- Monte Carlo methods 325 

— quasi-Monte Carlo methods 325 

selective node elimination method 137 

semi-implicit backward differentiation formula 
(BDF) scheme 254 

semi-stability 349 

shape parametrization 259 

simulation-free projection 34 

single-input single-output (SISO) system 123, 
243 

singular perturbation approximation 347 

singular value decomposition (SVD) 5, 63, 246, 
288, 332, 336, 339 

small gain condition 368 

Sobolev space 186 

Software-in-the-loop 402 

solid-extension mesh moving techniques 260 

Spalart-Allmaras (SA) turbulence model 205 

sparse identification of nonlinear dynamics 
(SINDy) 282, 299 


sparse regression 282, 299 

spectral element solver 284 

spectralembedding 297 

split congruence transformation 123 

Stability-Preserving, Adaptive Rational Krylov 
algorithm (SPARK) 455 

stable manifold theorem 285 

state-space system 390 

static condensation method 192 

statistical energy analysis (SEA) 76 

stochastic collocation technique 329, 337 

stochastic crack growth 415 

stochastic Galerkin method 322, 328, 337 

structure-preserving reduced-order interconnect 
macromodeling (SPRIM) method 128, 132 

subspace-angle interpolation method 216 

successive constraint method (SCM) 454 

supercritical Andronov—Poincaré—Hopf 
bifurcation 285, 301 

synchronization 352 

system identification (System ID) 296, 299, 310 


thermal energy equation 390 

time stepper Arnoldi algorithm 285 
Tonti diagram 150 

transfer function 121 

-admittance 126 

-impedance 126 

transfer matrix 

-admittance 155, 158, 159, 173 
-impedance 155, 158 

transmissions 407 

truncated balanced realization (TBR) 132 
trust region filter (TRF) approach 12, 13, 24 
trust region-based method 6 

— unconstrained trust region method 10 


uncertainty quantification (UQ) 203, 322 
unstable manifold theorem 285 


vector fitting method 137 

virtual controller 400 

virtual sensor 389, 405 

voltage-controlled voltage source (VCVS) 117 


weak greedy algorithm 214 
weighted adjacency matrix 350 
Whitney elements 189 


